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ABSTRACT 


We  describe  a  dual-process  theory  of  how  individuals  estimate  the  probabilities  of  unique  events, 
such  as  Hillary  Clinton  becoming  US  President.  It  postulates  that  uncertainty  is  a  guide  to 
improbability.  In  its  computer  implementation,  an  intuitive  system  1  simulates  evidence  in 
mental  models,  and  fonns  analog  non-numerical  representations  of  the  magnitude  of  degrees  of 
belief.  This  system  has  minimal  computational  power,  and  combines  evidence  using  a  small 
repertoire  of  primitive  operations.  It  resolves  the  uncertainty  of  divergent  evidence  for  single 
events,  for  conjunctions  of  events,  and  for  inclusive  disjunctions  of  events,  by  taking  a  primitive 
average  of  non-numerical  probabilities.  It  computes  conditional  probabilities  in  a  tractable  way, 
treating  the  given  event  as  evidence  that  may  be  relevant  to  the  probability  of  the  dependent 
event.  A  deliberative  system  2  maps  the  resulting  representations  into  numerical  probabilities. 
With  access  to  working  memory,  it  carries  out  arithmetical  operations  in  combining  numerical 
estimates.  Experiments  corroborated  the  theory’s  predictions.  Participants  concurred  in 
estimates  of  real  possibilities.  They  violated  the  complete  joint  probability  distribution  in  the 
predicted  ways,  when  they  made  estimates  about  conjunctions:  P(A),  P(B ),  P(A  and  B), 
disjunctions:  P(A),  P(B),  P(A  or  B  or  both),  and  conditional  probabilities  P(A),  P(B),  P(B\A). 
They  were  faster  to  estimate  the  probabilities  of  compound  propositions  when  they  had  already 
estimated  the  probabilities  of  each  of  their  components.  We  discuss  the  implications  of  these 
results  for  theories  of  probabilistic  reasoning. 
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1.  Introduction 


Probabilistic  thinking  is  ubiquitous  in  both  numerate  and  innumerate  cultures.  Aristotle 
wrote:  “A  probability  is  a  thing  that  happens  for  the  most  part”  (Aristotle,  Rhetoric,  Book  I, 
1357a35,  see  Bames,  1984).  His  account,  as  Franklin  (2001,  p.  109)  argues,  dominated 
philosophical  and  legal  thinking  about  probabilities  during  the  subsequent  two  millennia  until  the 
invention  of  the  probability  calculus.  It  continues  to  describe  probabilistic  thinking  in 
contemporary  contexts:  physicians  tend  to  offer  prognoses  using  qualitative  expressions,  e.g.,  “a 
high  risk”  and  “a  low  likelihood”  rather  than  quantitative  ones  (White  et  ah,  2010).  The  calculus 
transformed  the  qualitative  conception  of  probability  into  a  numerical  one.  It  depends  on  logic 
or  set  theory,  as  evinced  in  its  axiomatizations  (Scott  &  Krauss,  1966)  and  in  Boole’s  (1854)  dual 
interpretation  of  formulas  in  his  algebra  as  statements  about  sets  and  about  numerical 
probabilities.  Theorists  accordingly  treat  probabilities  as  estimates  of  truth  values,  and  as  a 
natural  extension  or  generalization  of  two-valued  logic  (e.g.,  de  Finetti,  1936/1995;  Jeffrey, 

1983;  Ramsey,  1926/1990). 

The  applicability  of  the  probability  calculus  to  everyday  reasoning  is  controversial, 
because  no  consensus  exists  about  the  proper  interpretation  of  the  calculus,  and  hence  about  the 
situations  to  which  it  applies  (see  Baron,  2008,  Ch.  5;  Nickerson,  2004,  Ch.  1).  Philosophers  are 
divided  on  this  issue  (see  Hajek,  2003).  Likewise,  some  psychologists  defend  a  frequency 
interpretation  of  probabilities,  which  therefore  does  not  apply  to  unique  events  (e.g.,  Cosmides  & 
Tooby,  1996;  Gigerenzer,  1994),  whereas  other  defend  a  subjective  view  of  probabilities 
corresponding  to  degrees  of  belief,  which  therefore  does  apply  to  unique  events  (e.g.,  Chater, 
Tenenbaum,  &  Yuille,  2006;  Tversky  &  Kahneman,  1983).  Complex  epistemological  issues 
underlie  even  these  so-called  “Bayesian”  views  of  probability,  which  also  divide  into  objective 


interpretations  (e.g.,  Williamson,  2010)  and  subjective  interpretations  (e.g.,  Jeffrey,  2004).  We 
will  not  pursue  these  epistemological  issues,  and  simply  note  that  naive  individuals  -  those  who 
have  not  mastered  the  probability  calculus  -  appear  to  estimate  probabilities  in  several  ways. 
They  use  frequency  data  if  they  are  available,  they  adduce  relevant  physical  considerations  such 
as  that  dice  have  six  sides,  and  they  use  evidence  to  estimate  the  probabilities  of  unique  events. 
They  can  make  informal  non-numerical  estimates,  specific  numerical  estimates,  or  estimates  of 
the  range  or  interval  of  probabilities.  Their  estimates  rely  on  two  main  methods  (Tversky  & 
Kahneman,  1983).  Extensional  methods  depend  on  deducing  the  probability  of  an  event  from 
the  mutually  exclusive  ways  in  which  it  could  occur,  which  are  either  equiprobable  or  of  known 
probabilities.  These  methods  are  explicable  in  terms  of  mental  models  of  the  possibilities 
(Johnson-Laird,  Legrenzi,  Girotto,  Legrenzi,  &  Caverni,  1999).  Non-extensional  methods 
depend  on  inducing  the  probability  of  an  event  from  some  relevant  evidence,  index,  or  heuristic 
such  as  that  if  an  exemplar  is  typical  of  a  category,  it  is  likely  to  be  a  member  of  the  category 
(Tversky  &  Kahneman,  1983). 

The  distinction  between  the  probabilities  of  events  that  occur  with  some  frequency  and 
the  probabilities  of  unique  events  is  subtle.  The  probability  that  a  particular  throw  of  a  roulette 
ball  lands  on  a  red  number  can  be  inferred  from  the  frequency  that  it  has  done  so  in  the  past,  i.e., 
that  particular  ball  on  that  particular  roulette  wheel.  The  instance  is  a  clear  member  of  a  single 
set  of  cases  in  which  the  ball  lands  on  a  red  number,  whose  cardinality  can  be  compared  with  that 
of  the  set  in  which  the  ball  does  not  land  on  a  red  number.  In  contrast,  the  probability  that 
Hillary  Clinton  is  elected  US  President  in  2016  is  not  a  member  of  a  single  set  of  cases  that 
allows  a  definitive  inference  of  its  value.  Many  different  sets  could  be  relevant  -  women  elected 
President,  Democrats  elected  President,  Yale  graduates  elected  President,  and  so  on.  They  yield 


different  frequencies  of  the  outcome,  and  none  of  these  frequencies  is,  or  can  be,  definitive.  The 
present  article  concerns  inferences  about  unique  events,  such  as  Clinton’s  election  to  the 
Presidency,  and  we  define  a  unique  event  as  one  that  is  not  a  member  of  any  single  set  allowing  a 
definitive  inference  of  the  frequency  of  its  occurrence. 

Tversky  and  Kahneman  (e.g.,  1983)  pioneered  the  study  of  the  probabilities  of  unique 
events,  and  they  demonstrated  egregious  fallacies  in  inferring  them  from  hypothetical  scenarios. 
The  present  research  diverges  from  their  studies  in  three  ways.  First,  we  focus  on  the 
probabilities  of  real  possibilities  in  everyday  life  (cf.  Ungar  et  ah,  2012).  Second,  we  do  not  use 
scenarios  designed  to  elicit  the  use  of  heuristics,  but  instead  simply  pose  questions  to  the 
participants  about  real  possibilities,  such  as:  What  is  the  probability  that  the  music  industry  will 
embrace  a  new  and  more  relaxed  version  of  copyright  law  in  the  next  10  years?  Third,  and  most 
important,  the  goals  of  our  research  are  to  describe  a  model-based  theory  of  the  mechanisms 
underlying  estimates  of  the  probabilities  of  unique  events  and  compounds  of  them,  to  outline  its 
implementation  in  a  computational  model,  and  to  report  experimental  corroborations  of  its 
principal  predictions. 

A  profound  mystery  about  estimates  of  the  probabilities  of  unique  events  is  where  the 
numbers  come  from  and  what  detennines  their  magnitudes.  It  is  conceivable,  of  course,  that 
estimates  of  such  probabilities  are  unsystematic,  unprincipled,  and  meaningless  (cf.  Cosmides  & 
Tooby,  1996).  And  so  one  aim  of  our  empirical  studies  was  to  check  whether  or  not  this 
possibility  is  correct.  On  the  presupposition  that  it  would  not  be,  we  developed  a  theory  of  how 
people  make  such  estimates  using  mental  models,  and  we  implemented  this  theory  as  part  of  a 
unified  computational  model  of  logical  and  probabilistic  reasoning,  mReasoner.  This  model- 
based  theory  and  its  implementation  explain  how  individuals  infer  the  probabilities  of  unique 


events,  and  where  the  numbers  in  such  estimates  come  from.  The  theory  also  predicts  systematic 
violations  of  the  probability  calculus.  In  order  to  explain  these  violations,  however,  we  must  first 
describe  the  complete  joint  probability  distribution  (the  JPD)  and  the  concept  of  subadditivity. 
Given  two  propositions  and  their  respective  negations: 

1.  A:  It  will  snow  tomorrow. 

B:  It  will  freeze  tomorrow. 

there  are  four  possible  contingencies:  A&B,  A&^B,  -'A&B,  and  ~^A&^B,  where  “&”  denotes  a 
logical  conjunction,  which  is  true  if  and  only  if  both  conjuncts  are  true,  and  “-i”  denotes  logical 

negation,  and  so  -'A  is  true  if  and  only  if  A  is  false.  The  respective  probabilities  of  each  of  these 

contingencies  make  up  the  complete  joint  probability  distribution  (JPD)  for  the  propositions  A 
and  B.  The  JPD  generalizes  to  any  number  of  random  variables,  and  to  variables  that  are  not 
just  binary  but  that  have  multiple  values.  The  JPD  yields  the  probability  of  any  compound 
proposition  concerning  its  individual  events.  Suppose,  for  example,  the  JPD  of  the  events  above 
is  as  follows: 

2.  P(snows  &  freezes)  =  .4 
P(snows  &  freezes)  =  .3 
P^  snows  &  freezes)  =  .2 
P^  snows  &  -•  freezes)  =  .  1 

They  yield  the  probability  of  any  compound  of  the  individual  events  in  it,  e.g.,  the  conditional 
probability  that  it  snows  given  that  it  freezes  is  .4/(.4  +  .2)  =  .66,  and  the  converse  conditional 
probability  that  it  freezes  given  that  it  snows  is  .4/(.4  +  .3)  =  .57.  The  conditional  probability, 
P(A\B)  can  also  be  inferred  using  Bayes’s  theorem,  which,  given  P(B)  ±  0,  can  be  expressed  as: 


3.  P(A|B)  = 


P(B) 


But,  the  three  probabilities  on  the  right-hand  side  of  the  equation  in  (3)  are  just  one  way  to  fix  the 
values  of  the  JPD.  And  the  conditional  probability,  P(A|B)  can  be  inferred  from  the  JPD  however 
its  values  are  fixed. 

A  subadditive  function  is  one  for  which  the  value  for  the  whole  is  less  than  the  sum  of  the 
values  for  the  parts,  as  in  the  square  root  function  where  V(4  +  4)  <  a/4  +  a/4.  Probability, 
however,  is  not  subadditive,  i.e.,  P(A)  =  P(A&B)  +  P(A&^B).  Hence,  P(A&B)  must  be  less  than 
or  equal  to  P(A),  and  less  than  or  equal  to  P(B).  Estimates  that  violate  this  principle  are 
subadditive  and  are  known  as  “conjunction  fallacies”,  because  the  probability  of  the  conjunction 
is  then  greater  than  the  probability  of  one  of  its  conjuncts  (see,  e.g.,  Tversky  &  Kahneman, 

1983).  Conjunction  fallacies  yield  at  least  one,  and  possibly  two,  negative  probabilities  in  the 
JPD.  A  set  of  estimates  of  probabilities  can  also  be  superadditive,  yielding  a  JPD  that  sums  to 
less  than  1.  The  estimates  of  P(A),  P(B),  and  P(A&B)  cannot  be  superaddive,  but  the  addition  of 
a  fourth  estimate,  such  as  P(~'A&^B),  can  be.  Subadditivity  and  superadditivity  have  nonnative 
consequences.  Reasoners  who  produce  estimates  of  probabilities  that  are  subadditive  or 
superadditive  violate  the  probability  calculus.  Their  perfonnance  is  inational  and  not  coherent, 
because  in  principle  a  “Dutch  book”  can  be  made  against  them,  i.e.,  a  set  of  bets  in  which  they 
are  bound  to  lose  (see,  e.g.,  de  Finetti,  1936/1995;  Ramsey,  1926/1990). 

Suppose  someone  estimated  the  following  probabilities  for  snow  and  freezing: 

4.  P(A)  =  .7 
P(B)  =  .7 
P(A&B)  =  .3 

The  estimates  do  not  commit  the  conjunction  fallacy,  but  the  probability  of  the  conjunction  is  too 
small  and  so  the  estimates  yield  the  following  JPD: 


5.  P(A&B) 


.3 


P(A&-B)  =  .4 
P(-A&B)  =  .4 
P(-A&-B)=  -.1 

The  three  estimates  therefore  yield  a  negative  probability  in  the  JPD. 

Negative  probabilities  are  impossible  in  the  probability  calculus,  but  Dirac  (1942) 
introduced  them  in  his  quantum  probabilities,  and  some  psychologists  have  argued  for  their  use 
to  account  for  human  judgments  (Pothos  &  Busemeyer,  2013).  A  corollary  is  that  the 
“conjunction  fallacy”  ceases  to  be  a  fallacy,  and,  as  Oaksford  (2013)  argues,  the  notion  of 
rationality  as  the  avoidance  of  inconsistency  ceases  to  apply.  In  daily  life,  nothing  is  less 
probable  than  the  impossible,  and  so  the  number  of  negative  probabilities  in  the  JPD  provides  a 
metric  for  inferences  about  probabilities.  For  JPDs  based  on  two  possibilities  A  and  B,  the  metric 
has  three  values:  0  negative  probabilities  (a  consistent  JPD),  1  negative  probability  (one  violation 
of  the  JPD),  and  2  negative  probabilities  (two  violations  of  the  JPD).  This  metric  is  analogous  to 
a  measure  of  “semantic  coherence”  (Wolfe  &  Reyna,  2010),  but  the  latter  depends  on  P(A),  P(B), 
P(A&B),  and  P(AvB).  In  contrast,  the  present  metric  applies  to  any  set  of  estimates  that  fix  a 
JPD. 

The  model  theory  predicts  the  occurrence  of  systematic  violations  of  the  JPD  in  many 
different  sorts  of  judgment,  and  we  report  studies  corroborating  the  occurrence  of  such 
violations.  In  what  follows,  the  article  describes  pertinent  previous  studies,  the  model  theory  of 
what  people  are  computing  in  estimating  unique  probabilities,  and  the  computer  implementation 
of  how  they  make  such  computations.  It  derives  the  model  theory’s  principal  predictions,  and 
reports  three  experiments  that  corroborate  these  predictions  for  conjunctive,  disjunctive,  and 


conditional  probabilities.  Finally,  it  discusses  the  implications  of  the  results  for  theories  of 
reasoning. 

2.  Previous  studies 

Extensional  and  non-extensional  estimates  of  probabilities  show  that  naive  individuals 
violate  the  norms  of  the  probability  calculus,  i.e.,  they  commit  fallacies.  In  extensional  cases, 
theorists  have  proposed  various  accounts  of  how  to  predict  estimates  of  the  probability  of 
compound  assertions,  but  they  center  around  the  hypothesis  that  individuals  take  the  mean  of  the 
probabilities  of  the  conjuncts  or  some  function  thereof  (e.g.,  Fantino,  Kulik,  Stolarz-Fantino,  & 
Wright,  1997;  Wyer,  1976),  the  mean  of  the  probabilities  of  disjuncts  in  estimates  of  disjunctions 
(e.g.,  Bar-Hillel  &  Neter,  1993;  Tversky  &  Kahneman,  1983;  Young,  Nussbaum,  &  Monin, 

2007),  and  a  similar  procedure  in  estimates  of  conditional  probabilities  (Reyna  &  Brainerd,  1995, 
2008;  Wolfe  &  Reyna,  2010).  In  non-extensional  cases,  theorists  propose  accounts  based  on  the 
representativeness  of  the  conjunction  in  comparison  with  the  vignette  that  is  the  basis  of  the 
estimates  (Barbey  &  Sloman,  2007;  Fantino,  1998;  but  cf.  Wallsten,  Budescu,  Erev,  &  Diederich, 
1997;  Wallsten,  Budescu,  &  Tsao,  1997).  In  their  studies  of  representativeness,  Tversky  and 
Kahneman  (1983,  p.  306)  considered  the  hypothesis  that  individuals  estimate  the  probability  of  a 
conjunction  by  taking  the  average  of  the  probabilities  of  its  conjuncts.  Tversky  and  Koehler 
(1994)  proposed  “support”  theory  to  account  for  estimates  of  the  probabilities  of  disjunctions, 
which  tend  to  be  less  than  or  equal  to  the  sum  of  the  probabilities  of  their  disjuncts  (Rottenstreich 
&  Tversky,  1997).  Likewise,  Zhao,  Shah,  &  Osherson  (2009)  present  convincing  evidence  that 
individuals  do  not  compute  the  ratio  of  P(A&B)  to  P(B)  in  estimating  P(A\B)  for  real  future 
possibilities.  Some  authors  propose  instead  that  they  rely  on  the  Ramsey  test  (e.g.,  Evans,  2007; 


Gilio  &  Over,  2012)  -  a  possibility  to  which  we  return  later. 

Despite  these  studies,  three  main  gaps  in  knowledge  exist.  First,  no  account  exists  of  the 
mental  processes  underlying  the  participants’  estimates  of  the  probabilities  of  real  but  unique 
possibilities,  though  Wyer  (1976)  called  for  such  investigations  nearly  40  years  ago.  Second,  as 
a  consequence,  psychologists  have  no  account  of  the  mental  processes  that  yield  numerical 
probabilities  from  non-numerical  premises.  Third,  with  few  exceptions  (Over  et  ah,  2007;  Zhao 
et  ah,  2009),  psychologists  have  not  studied  estimates  of  the  probabilities  of  unique  events  that 
are  real  possibilities.  Hence,  we  cannot  be  certain  that  individuals  concur  in  their  estimates  of 
such  probabilities,  or  whether  they  are  likely  to  violate  the  JPD  in  making  them.  We  now 
consider  a  theory  designed  to  fill  these  gaps  in  knowledge. 

3.  A  model  theory  of  unique  probabilities  and  their  compounds 

We  describe  a  theory  (the  “model”  theory)  that  is  founded  on  the  notion  that  individuals 
build  mental  models  to  understand  discourse  and  make  inferences.  The  model  theory  of  the 
probability  of  unique  events  aims  to  take  into  account  previous  studies,  to  ground  them  in  terms 
of  the  familiar  distinction  between  intuitions  and  deliberations,  and  to  derive  new  predictions 
about  the  probabilities  of  unique  events.  A  preliminary  study  described  experiments  with 
conjunctions  (Khemlani,  Lotstein  &  Johnson-Laird,  2012),  but  we  now  present  a  comprehensive 
theory  that  also  deals  with  the  probabilities  of  disjunctions  and  conditional  probabilities.  In  this 
section,  we  focus  on  what  people  are  computing.  But,  in  the  section  that  follows  we  outline  a 
computer  implementation  of  the  algorithm  that  the  model  theory  postulates  as  underlying  human 
estimates  of  unique  probabilities. 

The  theory  postulates  that  estimates  of  the  probabilities  of  unique  events  depend  on 


intuitions  (system  1)  and  on  deliberative  reasoning  (system  2).  This  distinction  is  a  familiar  one 
in  dual-process  theories  of  judgment  and  reasoning  (see,  e.g.,  Evans,  2008;  Johnson-Laird,  1983, 
Ch.  6;  Kahneman,  2011;  Sloman,  1996;  Stanovich,  1999;  Verschueren,  Schaeken,  &  d’Ydewalle, 
2005;  Wason  &  Evans,  1975;  Wason  &  Johnson-Laird,  1970).  These  theories  have  antecedents 
in  Turing  (1939)  and  Pascal  (1966/1669).  Some  accounts  (Evans  &  Stanovich,  2013)  have  been 
criticized  as  not  empirically  refutable  (Keren,  2013),  and  as  not  distinguishable  from  accounts 
based  on  a  single  system  (Kruglanski,  2013).  One  prophylactic  to  both  these  problems  is  to 
implement  a  computer  model  of  the  two  systems  and  of  how  they  interact.  The  next  section  of 
the  article  (4)  presents  such  an  implementation,  which  distinguishes  between  the  two  systems  in 
tenns  of  their  computational  power.  Here,  however,  we  describe  the  model  theory  of  intuitions 
about  probabilities  (System  1)  and  of  deliberative  processes  concerning  them  (System  2). 

3.1.  Intuitions  about  probabilities 

Intuitions  about  probabilities  are  not  numerical,  but  reflect  the  Aristotelian  idea  that 
events  vary  from  impossible  to  certain,  with  various  intennediate  cases.  They  can  be  expressed 
informally  (Budescu,  Karelitz,  &  Wallsten,  2003;  Wallsten  &  Budescu,  1995)  in  such 
Aristotelian  terms  as: 

6.  impossible,  almost  impossible,  highly  unlikely,  unlikely,  as  likely  as  not,  likely,  highly 
likely,  almost  certain,  certain. 

The  theory  postulates  that  the  single  most  important  intuitive  guide  to  improbability  is 
uncertainty.  The  two  seem  to  be  synonymous,  but,  as  we  show,  they  can  diverge  from  one 
another,  and  lead  to  errors  in  estimates  of  probabilities. 

Uncertainty  as  a  state  of  mind  arises  in  various  ways.  Suppose,  for  instance,  you  assess 


the  probability  of  the  following  event: 

7.  Hillary  Clinton  is  elected  US  President  in  2016. 

You  may  adduce  several  pieces  of  evidence  that  concur,  in  which  case  you  can  make  a 
corresponding  intuitive  estimate,  such  as,  “It  is  likely”.  But,  suppose  instead  that  your  evidence 
conflicts.  On  the  one  hand,  for  example,  you  think  that  Clinton  is  well  known  as  a  highly 
competent  politician,  and  that  many  people  will  vote  for  such  a  politician.  But,  on  the  other 
hand,  you  think,  she  is  female  and  no  woman  has  ever  been  elected  President.  Such  conflicts  are 
a  source  of  uncertainty:  one  piece  of  evidence  suggests  that  her  election  is  likely,  but  another 
piece  of  evidence  suggests  that  it  is  unlikely.  Your  problem  is  to  reconcile  at  least  two 
conflicting  judgments.  It  is  analogous  in  the  probability  calculus  to  inferring  P(Clinton-is- 
elected)  from  two  different  conditional  probabilities,  P(Clinton-is-elected\Clinton-is-competent) 
and  P(Clinton-is-elected\Clinton-is-female).  In  fact,  these  two  values  do  not  fix  P(Clinton-is- 
elected),  which  calls  for  still  more  evidence.  Normative  theories  of  how  to  reconcile  conflicting 
probabilities  can  be  founded  on  Bayesian  considerations  (e.g.,  Lindley,  Tversky,  &  Brown, 

1979).  But,  unless  you  are  an  expert,  your  intuitions  are  unlikely  to  suggest  a  need  for  additional 
evidence,  because  you  need  to  reconcile  the  current  conflict,  which  further  evidence  might 
exacerbate.  Intuition  yields  a  simple  conservative  compromise:  you  split  the  difference  between 
your  two  judgments,  i.e.,  you  take  their  average  (see  Wallsten,  et  al.,  1997).  Clinton’s  election  is 
accordingly,  “as  likely  as  not”.  Of  course  you  may  adduce  more  than  two  pieces  of  evidence, 
and  have  to  reconcile  a  conflict  between  them.  The  theory  postulates  that  system  1  deals  with 
them  by  reconciling  its  initial  estimate  of  a  probability  with  each  successive  piece  of  evidence, 
and  the  next  section  describes  a  feasible  mechanism. 


Suppose  you  have  to  estimate  the  probability  of  a  conjunction,  such  as: 


8.  Clinton  is  elected  US  President  in  2016  and  all  wars  in  the  world  cease  in  2017. 

If  the  evidence  you  adduce  suggests  that  Clinton  is  likely  to  be  elected,  and  other  evidence  - 
including  perhaps  the  election  of  Clinton  -  suggests  all  wars  are  not  likely  to  cease,  you  need  to 
resolve  the  conflict.  And,  once  again,  intuition  makes  a  compromise  and  takes  the  average. 

Disjunctions  are  a  major  source  of  uncertainty  both  in  reasoning  (e.g.,  Johnson-Laird, 
Byme,  &  Schaeken,  1992;  Johnson-Laird,  Lotstein,  &  Byrne,  2012)  and  in  decision  making 
(e.g.,  Tversky  &  Shafir,  1992).  An  assertion  about  an  individual  event,  such  as  (7),  rules  out 
more  possibilities  than  an  inclusive  disjunction  into  which  it  enters,  such  as: 

9.  Clinton  is  elected  US  President  in  2016,  or  all  wars  in  the  world  cease  in  2017,  or  both. 
This  disjunction  yields  three  separate  mental  models  to  represent  each  of  the  three  possibilities  to 
which  it  refers,  and  they  include  one  that  is  incompatible  with  the  model  of  the  individual  event, 
namely,  the  possibility  in  which  Clinton  is  not  elected,  but  all  wars  cease  in  2017.  It  follows  that 
the  assertion  about  the  individual  event  conveys  more  semantic  information  than  the  disjunction 
(Bar-Hillel,  1964).  Hence,  because  uncertainty  suggests  improbability,  intuition  assesses  the 
disjunction  as  less  likely  than  the  individual  event. 

Suppose  that  you  are  asked  to  estimate  a  conditional  probability,  such  as 

10.  Given  that  all  wars  in  the  world  end  in  2015,  what  is  the  probability  that  Clinton  is 
elected  US  President  in  November  in  2016? 

System  1  cannot  compute  Bayes’s  theorem,  and  so  the  model  theory  posits  that  intuition  relies  on 
a  simple  procedure.  It  treats  the  event  in  first  clause  in  (10)  as  evidence  for  the  event  in  the 
second  clause.  It  then  uses  the  same  procedure  that  we  described  earlier  to  estimate  the 
probability  of  an  individual  event  from  evidence.  According  to  Bayes’s  theorem,  a  conditional 
probability,  P(B\A),  depends  on  the  values  of  three  variables:  the  prior  converse  conditional 


probability,  P(A\B),  and  the  base  rates  of  the  two  events:  P(A)  and  P(B).  Equivalently,  P(B\A ) 
depends  on  the  ratio,  P(A&B)/P(A).  Naive  individuals,  according  to  the  theory,  tend  to  reduce 
the  problem  to  a  matter  of  how  the  dependent  variable,  P(B),  changes  given  that  the  independent 
variable  A  is  true.  Of  course,  individuals  may  suppose  that  the  given  event  in  (10)  is  irrelevant 
to  Clinton’s  election.  In  which  case,  they  can  adduce  their  own  evidence  for  her  election,  using 
the  procedure  for  individual  events.  A  conditional  probability  P(B\A)  in  the  standard  probability 
calculus  is  undefined  if  the  given  condition  A  is  false,  because  the  ratio  P(A&B)  to  P(A)  would 
then  call  for  division  by  zero.  But,  the  model  theory  allows  that  individuals  can  envisage 
counterfactual  models  (Byrne,  2005),  and  so  they  can  make  estimates  of  conditional  probabilities 
in  this  case  too.  Another  way  to  avoid  this  problem  is  to  formulate  the  probability  calculus  so 
that  conditional  probability  is  treated  as  a  primitive  rather  than  introduced  by  definition  (see  also 
Appendix  2  of  Adams,  1998). 

A  consequence  of  the  intuitive  methods  for  estimating  the  probabilities  of  compounds  is 
that  when  system  2  transfonns  system  1  ’s  intuitions  into  numerical  values,  the  resulting  values 
violate  the  complete  joint  probability  distribution  (the  JPD),  i.e.,  it  contains  at  least  one  negative 
probability.  Even  expert  probabilists  are  prone  to  violate  the  JPD  in  judgments  about 
probabilities.  Only  certain  special  cases  of  variables  are  likely  to  be  free  from  risks  of  violation. 
For  example,  if  one  event  is  inconsistent  with  another,  as  in: 

11.  Hillary  Clinton  is  elected  US  President  in  2016  and  Joe  Biden  is  elected  US  President  in 

2016 

individuals  know  that  it  is  impossible  for  both  events  to  occur,  and  so  they  will  estimate  the 
probability  of  their  conjunction  as  zero  regardless  of  their  estimates  of  the  probability  of  the 


conjuncts.  Likewise,  if  one  event  contradicts  another,  then  individuals  who  evaluate  their 
disjunction,  such  as: 

12.  Clinton  is  either  elected  US  President  in  2016  or  not 

know  that  one  or  other  of  these  two  is  bound  to  occur,  and  so  they  will  estimate  the  probability  of 
their  disjunction  as  1.0  regardless  of  their  estimate  of  the  probability  of  Clinton's  election.  But, 
contradictory  and  tautological  combinations  of  events  such  as  the  preceding  examples  are  rare, 
and  so  most  estimates  of  the  probabilities  of  unique  events  depend  on  evidence. 

3.2  Deliberations  and  numerical  probabilities 

The  deliberative  system  (system  2)  has  access  to  working  memory,  and  so  it  can  carry  out 
recursive  processes,  such  as  counting  and  other  arithmetical  operations  -  at  least  until  they 
overload  processing  capacity  and  call  for  an  external  notation.  And  it  is  this  system  that  maps 
the  intuitive  system’s  judgments  into  numerical  estimates.  The  deliberative  system  can  also  try 
to  keep  track  of  the  JPD,  though  naive  individuals  are  unlikely  to  do  so  in  any  exact  way  unless 
prompted.  One  corollary  of  the  difference  between  the  two  systems  concerns  the  apparent 
vagueness  of  verbal  descriptions  of  probabilities  in  comparison  with  numerical  probabilities.  In 
fact,  verbal  descriptions  make  explicit  the  contrast  in  polarity  between  affirmatives  such  as 
“probable”  and  negatives  such  as  “improbable”  around  a  neutral  midpoint  of  “as  likely  as  not”, 
whereas  numerical  probabilities  make  no  such  explicit  morphological  contrast.  Hence,  verbal 
descriptions  are  more  explicit  about  polarity  than  degree  whereas  numerical  probabilities  have 
the  converse  property  (see  Teigen  &  Brun,  2000).  Verbal  descriptions  of  the  sort  that  system  1 
produces,  such  as  “highly  probable”,  have  moderately  coherent  mappings  into  numerical 
estimates  (see,  e.g.,  Reagan,  Mosteller,  &  Youtz,  1989;  Wallsten  &  Budescu,  1995),  and  so  too 


do  determiners  such  as  “some”  and  “not  all”  (Teigen  &  Brun,  2003).  But,  if  the  numerical  scale 
is  of  a  finer  grain  than  intuitive  judgments,  the  mapping  calls  for  an  arbitrary  choice  of  a 
particular  number  from  a  range  of  possibilities.  Hence,  the  mapping  from  intuitions  to  a  scale 
should  be  more  likely  to  yield  a  consistent  JPD  with  a  coarse  scale  than  with  a  fine  scale.  Monte 
Carlo  simulations  bear  out  this  phenomenon,  and  a  previous  study  corroborated  this  prediction 
(Khemlani,  et  ah,  2012):  coarse  scales  yielded  a  greater  proportion  of  consistent  JPDs  than  fine 
numerical  scales,  and  this  result  supports  system  1  ’s  use  of  a  coarse  mental  scale. 

System  2  allows  individuals  to  hold  numerical  estimates  in  memory.  It  is  particularly 
useful  for  extensional  judgments  for  which  the  input  is  numerical  estimates  (see  John  son -Laird  et 
ah,  1999).  If  the  arithmetic  is  easy,  then  whether  the  data  are,  say,  natural  frequencies  or  chances 
out  of  100  -  does  not  seem  to  affect  estimates.  Readers  may  be  tempted  to  assume  that  system  2 
is  fully  rational.  In  fact,  it  suffers  from  several  impediments.  One  is  the  computational 
intractability  of  most  rational  systems,  including  the  probability  calculus,  logic,  and  decision 
theory.  Each  time  a  new  variable  is  added  to  the  JPD,  the  number  of  contingencies  it  contains 
doubles  in  size,  e.g.,  the  JPD  for  three  binary  variables  A,  B,  and  C,  represents  8  contingencies, 
and,  in  general,  n  binary  variables  calls  for  2n  contingencies.  Bayesian  nets  (Pearl,  1988),  which 
allow  one  to  infer  probabilities  without  having  to  construct  a  complete  JPD,  are  also  intractable, 
even  with  only  approximate  calculations  (Dagum  &  Luby,  1993).  Some  theorists  reject 
intractable  accounts  of  cognition  (e.g.,  Gigerenzer,  Hoffrage,  &  Goldstein,  2008;  Oaksford  & 
Chater,  2007,  Ch.  2);  other  theorists  propose  accounts  that  are  intractable  (for  a  review,  see  van 
Rooij,  2008).  A  synthesis  of  these  various  standpoints  is  that  human  reasoners  are  bound  to 
make  inferences  in  domains  that  cannot  have  tractable  algorithms,  such  as  two-dimensional 
spatial  reasoning  (Ragni,  2003).  In  consequence,  they  can  cope  only  with  small-scale  problems 


in  intractable  domains  (Johnson-Laird,  2006,  p.  340;  van  Rooij,  2008,  p.  971). 

Another  impediment  to  rationality  in  system  2  is  a  consequence  of  a  central  principle  of 
the  model  theory:  mental  models  represent  what  is  true  at  the  expense  of  what  is  false.  A 
corollary  is  that  system  2  deliberations  about  extensional  probabilities  can  err.  Consider,  for 
instance,  the  following  problem: 

13.  There  is  a  box  in  which  there  is  at  least  a  red  marble,  or  else  there  is  a  green 
marble  and  there  is  a  blue  marble,  but  not  all  three  marbles.  What  is  the 
probability  of  the  following  situation: 

In  the  box  there  is  a  red  marble  and  a  blue  marble? 

The  premise  has  two  mental  models:  one  represents  the  presence  of  a  red  marble,  and  the  other 
represents  the  presence  of  a  green  and  a  blue  marble.  In  neither  case  is  there  a  red  marble  and  a 
blue  marble.  As  this  account  predicts,  the  majority  of  participants  in  an  experiment  inferred  that 
the  probability  of  their  conjunction  was  zero  (Johnson-Laird  et  ah,  1999).  In  fact,  when  the  first 
disjunct  in  the  premise  is  true  (a  red  marble  is  in  the  box),  there  are  three  ways  in  which  the 
second  disjunct  can  be  false,  and  so  the  box  can  contain  any  of  the  following  four  sets  of 
marbles: 

red  green 

red  blue 

red 

green  blue 

The  second  of  these  possibilities  shows  that  a  red  and  blue  marble  can  be  in  the  box,  and  so  their 
conjunction  is  not  impossible.  Despite  the  occurrence  of  systematic  fallacies  of  this  sort,  system 
2  should  be  much  better  than  system  1  in  inferring  probabilities:  deliberation  trumps  intuition  in 


this  domain.  Inferences  about  the  probabilities  of  unique  events  cannot  be  estimated  in  a 
thoroughgoing  extensional  way,  and  so  human  reasoners  are  bound  to  rely  heavily  on  system  1 . 

A  major  impediment  to  the  rationality  of  system  2  is  ignorance.  Experts  can  rely  on 
system  2  to  use  the  numerical  probabilities  of  individual  events  to  compute  the  correct  values  for 
their  conjunction: 

14.  P(A&B)  =  P(A)*P(B|A) 
their  disjunction: 

1 5 .  P(AvB)  =  P(A)  +  P(B)  -  P(A&B) 

and  their  conditional  probability  using  Bayes’s  theorem  (3).  Naive  individuals,  in  contrast,  do 
not  know  these  formulas  for  computing  probabilities.  When  system  2  is  engaged,  they  can  at 
best  grasp  that  the  probability  of  a  conjunction  of  independent  events  is  their  product,  and  that 
the  probability  of  a  disjunction  of  inconsistent  events  is  the  addition  of  their  probabilities.  But, 
they  do  not  know  Bayes’s  theorem  (Zhao  et  ah,  2009),  and  even  when  they  are  taught  it,  they 
find  it  difficult  to  use,  because  it  is  not  intuitive  (Shimojo  &  Ichikawa,  1989).  We  polled  thirty 
cognitive  scientists  who  had  published  on  probability,  and  asked  them  the  following  question:  If 
one  knows  the  probability  of  A  and  the  probability  of  B,  what  constraints  if  any  do  they  place  on 
the  conditional  probability  of  B  given  A1  Some  replied,  “none”;  some  replied  that  they  did  not 
know;  and  only  a  handful  knew  the  correct  answer  (see  Appendix  A).  The  constraints  are  way 
beyond  the  intuitive  system;  and  estimates  of  conditional  probabilities  lie  on  the  boundary  of 
naive  competence  (Johnson-Laird  et  ah,  1999). 

The  best  definition  of  a  conditional  probability,  P(B\A),  according  to  Bennett  (2003)  is 
provided  by  Ramsey’s  (1926/1990)  test  for  fixing  your  degree  of  belief  in  assertions  of  the  form 
If  A  then  B:  you  add  A  to  your  beliefs  and  then  estimate  the  probability  of  B.  Evans  and  his 


colleagues  have  defended  the  Ramsey  test  as  a  psychological  account  of  how  individuals 
estimate  conditional  probabilities  (e.g.,  Evans,  2007,  p.  54).  Earlier  we  mentioned  the  problems 
of  computing  conditional  probabilities  when  the  given  condition,  A,  is  false.  The  Ramsey  test 
can  be  extended  to  deal  with  these  cases.  Individuals  adjust  their  beliefs  to  accommodate  its 
falsity  without  inconsistency,  and  then  assess  the  probability  of  B  in  that  context.  One  drawback 
with  the  Ramsey  test  is  that  it  takes  no  account  of  the  probability  of  A.  Another  drawback  is  that 
no  one  has  described  its  underlying  mental  processes,  and  any  plausible  account  of  them  is  likely 
to  be  computationally  intractable  (see  Oaksford  &  Chater,  2007,  p.  107).  In  contrast,  the  model 
theory,  as  we  described  earlier,  postulates  a  simple  tractable  procedure  for  estimating  conditional 
probabilities  in  system  1 .  We  now  turn  to  an  account  of  a  computer  implementation  of  the  model 
theory. 

4.  How  people  compute  unique  probabilities:  an  algorithm 

In  the  present  section,  we  describe  an  algorithmic  theory  of  the  mental  processes 
underlying  the  estimates  of  the  probabilities  of  unique  events.  Empirical  results  place  fewer 
constraints  on  algorithms  than  on  the  functions  they  compute,  because  any  computable  function 
has  a  countably  infinite  number  of  distinct  algorithms  (Rogers,  1967,  p.  9).  Nevertheless,  the 
success  of  the  model  theory’s  predictions,  as  shown  in  the  experimental  results  below,  lends 
some  support  to  the  present  account  of  how  the  computations  might  be  carried  out.  The  aim  of 
the  implementation,  however,  was  to  explain  in  principle  how  numerical  estimates  derive  from 
non-numerical  infonnation  of  an  Aristotelian  sort.  The  program,  mReasoner,  is  intended  to  unify 
deductive  and  probabilistic  reasoning,  and  it  embodies  all  of  the  operations  of  system  1  and 
system  2  for  the  probabilities  of  unique  events.  It  is  also  capable  of  many  other  sorts  of 


inference.  Its  source  code  is  in  the  public  domain,  and  is  available  at: 

http://mentalmodels.princeton.edu/models/mreasoner/.  The  reason  for  integrating  the  deductive 
and  probabilistic  systems  in  a  single  program  is  to  show  that  inferences  about  probabilities, 
whether  extensional  or  not,  can  be  made  with  mental  models.  The  program  copes  only  with  a 
fragment  of  natural  language,  and  it  does  not,  at  present,  deal  with  relations  of  the  following  sort: 
A  is  not  less  probable  than  B,  which  play  a  foundational  role  in  de  Finetti’s  theory  (Hosni,  2013). 
Table  1  summarizes  its  overall  plan. 

The  algorithm  is  based  on  three  main  constraints.  The  first  is  that  the  intuitive  system  has 
no  access  to  working  memory,  and  so  it  cannot  carry  out  recursive  or  iterative  processes.  Hence, 


Table  1 :  A  summary  of  mReasoner’s  Systems  1  and  2  for  inferring  the  probabilities  of  unique 
events  and  their  compounds. 


System  1 

Assessment  of  the  probability  of 
unique  events 

1.  Represent  a  piece  of  evidence  as  a 
mental  model  (Khemlani  &  Johnson- 
Laird,  2012a,  b). 

2,  Convert  a  mental  model  into  an  icon 
representing  a  probability:  magnitudes 
in  icons  correspond  to  probabilities. 

3,  For  cases  in  which  two  conflicting 
estimates  occur,  use  a  primitive  average 
to  resolve  the  conflict  (see  Table  2). 

4.  Translate  the  resulting  icon  into 
informal  language,  e.g.,  “highly 
probable”. 


System  2 

Numerical  and  arithmetical 
computations 

5.  Convert  an  icon  (from  3)  into  a 
numerical  probability. 

6.  Carry  out  arithmetical  operations  in 
extensional  reasoning  about  the 
probability  of  compounds  (Johnson- 
Laird  et  ah,  1999). 

7.  Check  the  consistency  of  the  JPD, 
where  prompted  to  do  so,  for  simple 
cases. 


Note:  Steps  1-4  can  be  used  to  compute  compound  probabilities,  e.g.,  P(A&B).  Conjunctive  and 
disjunctive  probabilities  use  primitive  averaging  to  estimate  the  compound,  while  conditional 
probabilities,  P(A\B)  use  B  as  evidence  to  estimate  P(A). 


it  cannot  carry  out  arithmetical  operations,  such  as  counting  and  addition.  It  lacks  even  the 


computational  power  of  finite-state  automata  (Hopcroft  &  Ullman,  1979),  because  it  can  carry 
out  a  loop  of  operations  for  only  a  small  number  of  times  -  a  restriction  that  is  built  into  its 
computer  implementation  in  mReasoner. 

The  second  constraint  is  that  the  algorithm  adduces  as  little  evidence  as  possible  in  order 
to  estimate  probabilities.  But,  additional  evidence  can  push  the  probability  represented  in  the 
system  one  way  or  the  other.  If  a  piece  of  evidence  concurs  with  the  current  estimate,  the  system 
yields  the  informal  estimate  that  they  both  support.  If  a  piece  of  evidence  conflicts  with  the 
current  estimate,  then  the  system  resolves  the  conflict.  It  can,  in  principle,  add  further  evidence 
in  a  piecemeal  way,  resolving  any  conflict  between  its  current  estimate  and  the  impact  of  the  new 
piece  of  evidence. 

The  third  constraint  is  that  the  algorithm  relies  on  primitive  analog  representations  of 
magnitudes  akin  to  those  in  animals  (Meek  &  Church,  1983),  infants  (Barth,  et  ah,  2006;  Carey, 
2009;  Dehaene,  1997),  and  adults  in  numerate  (Song  &  Nakayama,  2008)  and  non-numerate 
cultures  (Gordon,  2004).  These  icons  are  not  numerical,  and  they  represent  magnitudes  by  a 
direct  analogy:  the  greater  the  size  of  the  icon,  the  greater  the  magnitude  that  it  represents.  For 
probabilities,  icons  represent  the  degree  of  belief  in  a  proposition  as  a  pointer  on  a  Tine’ 
extending  between  an  origin  representing  impossibility  and  a  maximum  representing  certainty: 

16.  | -  I 

The  left  vertical  represents  impossibility,  the  right  vertical  represents  certainty,  and  the  mid-point 
between  them  represents  the  boundary  between  the  probable  and  the  improbable,  though  for 
simplicity,  we  omit  midpoints  from  these  diagrams.  The  rightmost  end  of  the  line  is  a  pointer 
that  represents  a  magnitude  corresponding  to  the  strength  of  a  particular  belief,  i.e.,  a  subjective 
probability.  The  line  is  not  the  real  number  continuum,  which  is  infinitely  and  non-denumerably 


dense,  and  so  cannot  have  an  isomorphic  representation  in  any  finite  device,  such  as  the  human 
brain.  Instead,  the  line  has  granularity  -  it  is  made  up  of  minimal  indivisible  elements.  The 
model  theory  postulates  that  the  granularity  of  mental  icons  is  quite  coarse,  and  mReasoner  uses 
similarly  coarse  icons  with  a  granularity  of  eight  divisions  between  impossibility  and  certainty, 
corresponding  to  the  informal  Aristotelian  terms  listed  in  (6)  above.  Overall,  system  1  adduces 
evidence  from  knowledge  of  a  relevant  domain,  simulates  it  in  mental  models,  and  uses  these 
models  to  fonn  an  icon,  which  it  can  describe  in  Aristotelian  terms. 

The  mReasoner  program  can  judge  probabilities  using  its  small  knowledge  base,  or  it  can 
interact  with  users  to  acquire  such  knowledge.  When  it  interacts  with  users,  its  first  step  is  to 
solicit  from  them  any  logical  relation  between  the  relevant  propositions,  A  and  B.  For  example, 
if  A  contradicts  B,  then  their  conjunction  is  impossible  and  has  a  probability  of  0,  but  their 
disjunction  is  certain  and  has  a  probability  of  1 .0.  If  instead  A  implies  B ,  then  their  conjunction 
equals  the  probability  of  A.  The  program  also  assesses  whether  the  user  believes  that  the 
occurrence  of  A  has  any  effect  on  the  probability  of  B,  and  whether  such  an  influence  increases 
or  decreases  its  value.  It  solicits  evidence  for  this  influence  from  the  user.  We  now  consider 
how  system  1  uses  evidence  to  infer  the  probabilities  of  unique  events,  and  how  it  infers 
conjunctive,  disjunctive,  and  conditional  probabilities. 

4.1.  The  implementation  of  intuitions  about  probabilities 

System  1  in  mReasoner  can  make  an  intuitive  estimate  of  a  unique  probability,  such  as: 

17.  What  is  the  probability  that  Apple  is  profitable  this  year? 

It  uses  knowledge,  such  as: 

18.  Apple  is  a  well-managed  company 


and: 


19.  Most  well-managed  companies  are  profitable, 
in  order  to  infer: 

20.  Apple  is  likely  to  be  profitable. 

The  preceding  inference  is  striking,  because  the  conclusion  refers  to  a  probability,  but  neither 
premise  does,  so  an  obvious  question  is:  where  does  the  probability  come  from?  System  1  infers 
it  from  a  mental  model  of  the  quantified  assertion  (19).  In  principle,  it  could  do  so  from  a  pre¬ 
existing  model  in  its  database,  but  the  present  method  allows  users  to  input  their  own  premises 
akin  to  those  above.  The  program  constructs  a  mental  model  representing  the  evidence  that  most 
well  managed  companies  are  profitable: 

2 1 .  well-managed-firm  profitable 
well-managed-finn  profitable 
well-managed-firm  profitable 
well-managed-finn 

This  diagram  denotes  a  mental  model  of  an  arbitrary  number  of  entities  -  four  in  this  case,  and  so 
its  first  row  represents  a  well-managed  company  that  is  profitable  (for  the  details  of  how  models 
are  constructed  from  quantified  assertions,  see  Khemlani  &  Johnson-Laird,  2012a,  b).  Unlike 
the  diagram,  a  real  mental  model  is  a  simulation  of  actual  entities  (Khemlani,  Mackiewicz, 
Bucciarelli,  &  Johnson-Laird,  2013).  Some  models,  however,  also  contain  abstract  symbols, 
e.g.,  a  symbol  for  negation  (Khemlani,  Orenes,  &  Johnson-Laird,  2012).  Given  that  Apple  is  a 
well-managed  firm,  system  1  uses  the  model  above  to  construct  an  icon  derived  from  the 
proportion  in  the  model  of  well-managed  firms  that  are  profitable: 


22.  | 


System  1  can  translate  this  icon  into  a  verbal  description: 

23.  It’s  highly  likely. 

System  1  copes  with  determiners  ranging  from  all  and  any  through  most  and  many  to  few 
and  none.  It  can  represent  only  a  single  mental  model  at  any  one  time,  but  it  can  consider  more 
than  one  piece  of  relevant  evidence.  Indeed,  users  may  know  that  Apple  is  an  innovator,  and 
believe  that  some  innovators  are  profitable.  If  system  1  uses  only  this  evidence,  and  ignores  the 
evidence  yielding  the  conclusions  in  (23),  it  constructs  the  following  icon: 

24.  | -  | 

This  icon  ends  at  the  midpoint,  and  so  it  translates  into:  “as  likely  as  not”. 

Readers  may  wonder  why  the  theory  postulates  the  use  of  an  icon  given  that  proportions 
in  models  correspond  to  what  the  icon  represents.  The  answer  is  that  an  icon  allows  different 
pieces  of  evidence  for  an  event  to  be  combined  in  order  to  estimate  its  probability.  And  so  system 
1  can  combine  the  effects  of  several  pieces  of  evidence  about  Apple's  profitability.  It  starts  with 
the  first  piece  of  evidence,  such  as  that  most  well  managed  companies  are  profitable,  from  which 
given  that  Apple  is  well  managed,  it  can  make  the  inference  yielding  the  belief  icon  in  (22). 

With  further  evidence,  such  as  that  Apple  is  an  innovator,  and  that  some  innovators  are 
profitable,  it  infers  that  Apple’s  profitability  is  “as  likely  as  not”.  It  inserts  this  probability  as  a 

second  pointer,  “a”,  into  the  existing  icon  in  (22)  to  yield  one  that  represents  two  different 

probabilities  (the  new  pointer  and  the  right-hand  end  of  the  icon): 

25.  | - A  —  | 

System  1  then  makes  a  primitive  average  of  the  pointer  and  the  end  of  the  icon  to  yield: 


26.  | 


The  two  pieces  of  evidence  about  a  single  event  yield  an  icon  supporting  the  conclusion  that 
Apple  is  likely  to  be  profitable  (20).  Likewise,  any  subsequent  piece  of  evidence  can  shift  the 
existing  pointer  using  the  same  mechanism.  The  theory,  however,  postulates  that  individuals 
tend  to  consider  only  a  minimal  amount  of  evidence. 

The  way  in  which  system  1  makes  primitive  averages  depends  on  its  non-numerical 
operations,  which  we  now  review  before  considering  the  computation  of  the  probabilities  of 
compound  events.  System  1  can  carry  out  six  non-numerical,  or  primitive,  operations:  p- 
averages,  p-adds,  p-subtracts,  p-longer,  p-multiplies,  and  p-divides.  Table  2  summarizes  these 
operations.  Each  of  them  relies  on  a  finite  loop  that  iterates  no  more  than  a  small  fixed  number 
of  times  depending  on  the  granularity  of  icons.  Both  the  number  of  iterations  and  the  granularity 
of  icons  may  vary  from  one  person  to  another.  The  operation,  p-averages,  is  a  primitive  fonn  of 
averaging:  it  moves  two  pointers  towards  one  another  until  they  meet,  and  so  it  yields  a  rough 
average  of  their  values.  This  operation  deals  with  icons  representing  conflicts  in  the  evidence  for 
a  single  event,  such  as  Apple’s  profitability.  It  is  also  the  simplest  operation  in  Table  2.  We 
assessed  the  simplicity  of  the  operations  in  Table  2  using  the  standard  measure  of  Kolmogorov 
complexity  (see  Li  &  Vitanyi,  1997).  This  measure  assesses  the  complexity  of  a  string  of 
symbols  from  the  length  of  its  shortest  possible  description  in  a  reference  language,  and  this 
measure  predicts  the  difficulty  that  naive  individuals  have  in  specifying  informal  algorithms 
(Khemlani  et  ah,  2013).  We  used  it  to  assess  the  complexity  of  the  corresponding  Lisp  code  that 
computes  the  functions  in  Table  2.  The  simplest  of  them  was  p-averages.  Slightly  more  complex 
operations  are  p-adds,  which  adds  two  magnitudes,  and  p-subtracts,  which  subtracts  a  smaller 
magnitude  from  a  larger  one.  The  results  of  the  operations  cannot  be  larger  than  the  icon 
representing  “certainty”,  or  smaller  than  the  empty  icon  representing  “impossibility”. 


The  operations  in  Table  2  underlie  intuitive  estimates  of  conjunctive,  disjunctive,  and 
conditional  probabilities,  which  we  described  earlier.  Conjunction,  P(A&B),  depends  on  p- 
averages  to  resolve  the  conflict  between  the  probabilities  of  the  two  conjuncts,  P(A)  and  P(B). 
The  system  assesses  the  probability  of  a  disjunction,  P(AvB),  guided  by  the  uncertainty  of 
disjunctions,  and  so  it  also  uses  p-averages.  Conditional  probability,  P(B\A),  uses  A  as  evidence 
for  B  in  the  same  way  that  system  1  uses  evidence  to  assess  the  probability  of  a  single  event, 
such  as  Apple’s  profitability.  If  the  evidence,  A,  neither  increases  nor  decreases  the  magnitude  of 
P(B),  the  system  returns  its  initial  estimate  of  P(B).  System  1  can  compute  elementary  logical 
relations  between  A  and  B ,  such  as: 

27.  Clinton  is  elected  US  President  in  2016  or  Clinton  is  not  elected  US  President  in  2016 
and  in  this  case  returns  the  value  of  “certain”  for  the  disjunction. 

In  summary,  system  1  in  mReasoner  infers  intuitive  non-numerical  probabilities  from 
mental  models  of  evidence,  which  it  represents  iconically  as  magnitudes  in  order  to  combine 
them.  Its  combinations  depend  on  primitive  analogs  of  averaging,  adding,  and  multiplying 
magnitudes  in  these  icons  in  order  to  infer  the  probabilities  of  compound  events.  System  1,  of 
course,  has  a  role  in  other  sorts  of  cognition,  where  it  is  susceptible  to  the  same  constraints  on  its 
operations.  For  example,  its  limitations  predict  the  intuitive  conclusions  that  individuals  tend  to 
draw  from  syllogistic  premises  (see  Khemlani,  Lotstein,  &  Johnson-Laird,  2014).  In  all  three 
probabilistic  cases  that  we  have  considered  -  conjunctive,  disjunctive,  and  conditional 
probabilities  -  the  output  of  system  1  is  an  icon  representing  an  intuitive  probability.  It  can  be 
put  into  qualitative  descriptions  using  Aristotelian  terms,  as  in  (6)  above,  or  it  can  be  passed  on 
to  system  2  in  order  to  be  expressed  numerically. 


Table  2:  The  five  primitive  arithmetical  operations  that  System  1  in  mReasoner  carries  out 


Primitive 

operations 

Input 

Operation 

Output 

p-averages 

One  icon  with  two 

magnitude 

pointers 

Finite  loop  moves  the  two  pointers  an 
increment  at  a  time  so  they  converge  on 
an  approximate  mid-point 

A  magnitude  that  is  an 
average  of  the  initial 
pointers'  magnitudes 

p-adds 

One  icon  with  two 

magnitude 

pointers 

Finite  loop  joins  each  element  of  second 
magnitude  to  the  first,  checks  result 
against  maximum  magnitude  using  p- 
subtracts. 

A  magnitude  < 

the  one  equivalent  to 

“certainty”. 

p-subtracts 

One  icon  with  two 

magnitude 

pointers 

Finite  loop  removes  each  element  of 
second  magnitude  from  the  first.  Halts  if 
first  magnitude  becomes  empty. 

A  magnitude  >  empty 
one  equivalent  to 
“impossibility”. 

p-longer 

One  icon  with  two 

magnitude 

pointers 

Finite  loop  p-subtracts  second 
magnitude  from  first. 

If  first  magnitude  is  not 
empty,  returns  “true”; 
otherwise  “nil”. 

p-multiplies 

One  icon  with  two 
non-empty 
magnitude 
pointers 

Finite  loop  p-adds  second  magnitude  to 
first,  and  p-subtracts  an  element  from 
second  on  each  iteration.  Halts  when 
second  becomes  empty.  Checks  result 
against  maximum  magnitude  using  p- 
subtracts. 

A  magnitude  < 
standard  one  equivalent 
to  “certainty”. 

p-divides 

One  icon  with  two 
non-empty 
magnitude 
pointers 

Finite  loop  p-subtracts  denominator 
magnitude  from  numerator  magnitude, 
p-adds  an  element  to  output  on  each 
iteration  until  numerator  is  p-longer 
than  denominator. 

A  magnitude  >  empty 
one  equivalent  to 
“impossibility”. 

4.2  The  implementation  of  deliberations  yielding  numerical  probabilities 

System  2  has  access  to  working  memory  and  so  it  can  carry  out  more  complex 
operations,  including  counting  and  arithmetic.  Its  role  in  probabilistic  inferences  is  likely  to  be 
confined  to  transforming  non-numerical  icons  into  numerical  probabilities,  and  carrying  out 
slightly  more  rational  operations,  such  as  the  exact  multiplication  of  probabilities  for 
conjunctions,  and  their  addition  for  disjunctions.  Multiplication,  however,  is  nonnatively  correct 


only  for  the  conjunction  of  independent  events,  and  addition  is  likewise  normatively  correct  only 
for  disjunctions  of  mutually  inconsistent  events.  The  version  of  system  1  implemented  in 
mReasoner  is  able  to  carry  out  primitive  analogues  of  these  operations  too.  Hence,  a  remaining 
open  question  is  whether  in  fact  these  operations  are  carried  out  by  system  2. 

4.3  The  general  predictions  of  the  model  theory 

The  model  theory  makes  four  general  predictions  about  estimates  of  the  probabilities  of 
unique  events: 

1.  Probability  estimates  of  unique  events  should  be  systematic:  individuals  from  the  same 
population  have  available  roughly  the  same  sorts  of  evidence  about  real  possibilities,  and 
so  their  estimates  should  concur  reliably  in  rank  order. 

2.  The  primitive  operations  of  system  1  should  lead  to  systematic  violations  of  the  JPD.  In 
the  case  of  a  conjunction,  intuition  makes  a  simple  compromise  between  conflicting 
values  of  P(A)  and  P(B),  yielding  their  primitive  average.  In  the  case  of  a  disjunction, 
intuition  takes  the  uncertainty  between  P(A)  and  P(B)  to  elicit  their  primitive  average.  In 
the  case  of  a  conditional  probability,  P(B\A),  intuition  treats  A  as  evidence  for  estimates 
of  P(B). 

3.  When  individuals  have  already  made  estimates  of  P(A)  and  P(B),  they  should  be  more 
likely  to  use  system  2’s  methods  to  estimate  conjunctive  and  disjunctive  probabilities. 
Hence,  they  should  tend  to  estimate  P(A&B)  as  the  product  of  P(A)  and  P(B),  which  is 
appropriate  only  for  independent  events;  and  they  should  tend  to  estimate  P(AvB)  as  the 
sum  of  P(A)  and  P(B),  which  is  also  appropriate  only  for  inconsistent  events.  They  have 
no  deliberative  method  for  estimating  conditional  probabilities  about  unique  events. 


4.  When  individuals  have  already  made  estimates  of  P(A)  and  P(B),  they  should  be  faster  to 
make  estimates  of  any  sort  of  compound. 

Although  some  studies  have  examined  estimates  of  the  probabilities  of  real  unique  possibilities 
(e.g.,  Zhao,  et  ah,  2009),  none  as  far  as  we  know  provides  results  that  allow  us  to  test  these 
predictions.  We  therefore  carried  out  three  experiments  to  do  so.  Three  different  ways  to  fix  the 
JPD  for  two  binary  variables  A  and  B  are  as  follows  (see  Appendix  B): 

1. P(A),  P(B),  and  P(A&B) 

2.  P(A),  P(B),  and  P(A  or  B  or  both) 

3.  P(A),  P(B),  and  P(B|A). 

A  challenge  to  readers  is  to  discover  a  set  of  three  judgments  of  probabilities  that  fix  the  JPD  for 
two  binary  variables  A  and  B ,  but  that,  no  matter  what  their  values  are,  these  values  cannot 
violate  the  JPD.  The  triple  can  include  probabilities  of  individual  events,  such  as  P(A)  or  P(B), 
and  any  sort  of  compound  probabilities,  such  as  P(A&B),  P(A  or  B  or  both),  or  P(A  \B).  We 
return  to  this  puzzle  at  the  end  of  the  paper.  However,  estimates  for  the  three  triples  above  can 
violate  the  JPD,  and  we  used  them  in  our  experiments  to  test  the  model  theory’s  predictions.  In 
each  study,  participants  estimated  the  probability  of  pairs  of  related  unique  events,  P(A)  and 
P(B).  In  Experiment  1,  participants  also  made  an  estimate  of  the  probability  of  their  conjunction, 
P(A  and  B);  in  Experiment  2,  they  made  an  estimate  of  the  probability  of  their  disjunction,  P(A 
or  B  or  both)',  and  in  Experiment  3  they  made  estimates  of  the  two  conditional  probabilities 
P(B\A)  and  P(A\B). 

5.  Experiment  1:  Conjunctions  of  unique  events 

When  naive  individuals  estimate  the  probability  of  a  conjunction  of  two  unique  events, 


the  model  theory  predicts  that  their  intuitions  should  be  based  on  the  primitive  average  of  the 
probabilities  of  the  two  conjuncts,  or,  in  deliberations,  they  should  be  based  on  multiplying  them. 
The  fonner  yields  violations  of  the  JPD;  the  latter  does  not,  but  it  is  appropriate  only  when  the 
two  events  are  independent  of  one  another. 

5.1  Method 

The  participants  each  had  to  estimate  sets  of  three  probabilities  of  unique  events  in  four  different 
orders: 

1.  P(A&B),  P(A),  P(B) 

2.  P(A),  P(A&B),  P(B) 

3.  P(B),  P(A&B),  P(A) 

4.  P(A),  P(B),  P(A&B) 

There  were  16  sets  of  materials,  which  concerned  unique  events  in  domains  such  as  politics, 
sports,  and  science  (Appendix  C  presents  the  full  set  of  materials).  For  half  the  materials  of  the 
problems,  A  was  likely  to  increase  the  value  of  P(B),  and  for  the  other  half,  A  was  likely  to 
decrease  the  value  of  P(B).  In  a  prior  norming  study,  judges  decided  independently  whether  A 
increased  P(B),  decreased  P(B),  or  had  no  effect  on  P(B)  for  a  larger  battery  of  materials.  For  the 
two  sorts  of  materials  in  the  present  experiment,  we  used  only  items  for  which  the  majority  of 
judges  had  categorized  A  as  increasing  P(A)  and  as  decreasing  P(B),  respectively.  Hence,  A  and 
B  in  the  experiment  were  not  independent  events.  The  allocation  of  the  materials  to  the  four 
orders  was  random  with  the  constraint  that  all  the  participants  had  four  trials  in  each  of  the  four 
orders,  which  were  presented  to  each  of  them  in  a  different  random  order.  The  problems  were 
presented  one  question  at  time  in  the  appropriate  order  on  the  computer  screen.  The  participants 


made  a  response  by  moving  a  slider  on  a  scale  from  0%  to  100%,  and  the  relevant  percentage 
probability  of  the  position  of  the  slider  appeared  above  it.  The  instructions  told  the  participants 
to  make  their  best  estimate  of  the  probabilities  of  the  events,  but  did  not  mention  that  their 
responses  would  be  timed. 

Forty-two  participants  completed  the  study  for  monetary  compensation  (a  $10  lottery)  on 
an  online  platform,  Amazon  Mechanical  Turk  (for  an  evaluation  of  the  validity  of  this  method, 
see  Paolacci,  Chandler,  &  Ipeirotis,  2010).  All  of  the  participants  stated  that  they  were  native 
English  speakers. 

5.2  Results  and  discussion 

The  participants  concurred  in  the  rank  order  of  their  estimates  over  the  different  materials 
for  P(A),  for  P(B),  and  for  P(A&B)  (Kendall’s  W’s  =  .32,  .16  and  .21,  respectively,  all  ps  < 
.0001).  These  significant  correlations  corroborated  the  model  theory’s  first  prediction  that 
individuals  rely  at  least  in  part  on  knowledge  and  processes  in  common  to  estimate  unique 
probabilities.  The  raw  data  for  this  experiment  and  the  two  subsequent  ones  are  available  at 
http://mentalmodels.princeton.edu/portfolio/unique-events/. 

Figure  1  shows  the  percentages  of  trials  for  each  of  the  16  materials  in  which  participants 
estimated  the  probability  of  the  conjunctions  by  taking  p-averages  of  the  probabilities  of  their 
conjuncts,  and  p-multiplies  of  the  probabilities  of  the  conjuncts.  We  scored  a  response  as  a  p- 
average  or  as  a  p-multiplies  provided  a)  that  it  was  within  12.5%  of  the  correct  arithmetic  value, 
which  assumes  a  granularity  of  eight  divisions  in  the  belief  icon;  and  b)  in  the  case  of  p-average, 
that  the  conjunctive  probability  estimate  was  in  between  the  estimated  probabilities  of  the 
individual  conjuncts.  The  balances  of  the  percentages  are  cases  in  which  the  participants  used 


other  miscellaneous  strategies  of  an  unknown  nature.  A  multiplicative  response  would  have  been 
normative  had  the  two  events  been  mutually  exclusive,  but  they  were  not.  Goodness  of  fit 
metrics  showed  that  a  p-average  of  each  participants’  estimates  of  P(A)  and  P(B),  on  a  trial  by 
trial  basis,  provided  a  better  fit  of  the  data  (Kolmogorov-Smirnov  test,  D  =  .43,  p  <  .001;  R  = 

■82,  p  <  .0001;  RMSE  =  .11)  than  a  p-multiplication  of  their  estimates  of  the  two  probabilities 
(Kolmogorov-Smirnov  test,  D  =  .45,  p  <  .0005;  R2  =  .78,  p  <  .0001;  RMSE  =  .14). 


Figure  1 :  The  percentages  of  trials  on  which  the  participants  in  Experiment  1  estimated  P(A&B) 
by  taking  a  p-average  or  a  p-multiplication  of  P(A)  and  P(B)  for  each  of  the  16  materials  in  the 
experiment  (see  Appendix  C).  The  balances  of  percentages  in  each  case  were  miscellaneous 
responses. 


Material  number 

But,  the  best  fit  with  the  data  was  a  p-average  for  those  estimates  on  which  JPDs  were  violated 
and  otherwise  their  p-multiplication  (Kolmogorov-Smirnov  test,  D  =  .14,  n.s.;  p  =  .79;  R“  =  .91, 
p  <  .0001;  RMSE  =  .05).  Overall,  28  of  the  42  participants  made  conjunctive  estimates  that  fell 
between  their  estimates  of  P(A)  and  P(B)  on  at  least  50%  of  trials  (Binomial  test,  p  <  .025, 


assuming  a  conservative  prior  probability  of  .5).  These  results  corroborate  the  model  theory’s 
second  prediction  that  individuals  should  make  systematic  violations  of  the  norms  of  the 
probability  calculus,  and  that  in  the  case  of  conjunctions  their  estimates  were  either  primitive 
averages  or  primitive  products  of  the  two  probabilities  (see  Table  2). 

Table  3  presents  the  percentages  of  trials  on  which  there  were  violations  of  the  JPD,  and 
the  mean  percentages  that  the  participants  estimated  for  the  three  sorts  of  events,  depending  on 
the  order  of  the  judgments.  As  it  shows,  violations  were  quite  marked  when  the  conjunction 
came  first  in  the  series  of  judgments.  Indeed,  order  had  a  highly  reliable  effect  on  the  frequency 
of  violations:  the  earlier  the  position  of  the  conjunction,  the  greater  the  likelihood  of  a  violation 
(Page’s  L  =  53 1 ,  z  =  2.94,  p  <  .003). 


Table  3.  The  percentages  of  trials  yielding  violations  of  the  JPD  in  Experiment  1,  and  the  mean 
percentage  judgments  for  three  sorts  of  events  depending  on  their  order. 


The  estimation  order 

The  percentages 
of  violations  of 
JPD 

P(A)  as  a 
percentage 

P(B)  as  a 
percentage 

P(A&B)  as  a 
percentage 

1 .  P(A&B)  P(A)  P(B) 

61 

41 

32 

30 

2.  P(A)  P(A&B)  P(B) 

49 

38 

34 

27 

3.  P(B)  P(A&B)P(A) 

51 

35 

38 

28 

4.  P(A)  P(B)  P(A&B) 

46 

38 

41 

29 

Overall  results 

50 

38 

37 

29 

The  percentages  of  estimates  in  which  the  participants  p-multiplied  P(A)  and  P(B)  to  estimate 
P(A&B)  were  as  follows: 

P(A&B)  P(A)  P(B)  25% 

P(A)  P(A&B)P(B)  26% 

P(B)  P(A&B)  P(A)  30% 


P(A)  P(B)P(A&B) 


42% 


The  trend  for  more  multiplicative  responses  the  later  in  the  series  the  estimate  of  P(A&B) 
occurred  was  reliable  (Page’s  L  =  534,  z  =  3.27,  p  =  .001,  collapsing  the  two  middle  orders).  This 
result  corroborates  the  model  theory’s  third  prediction. 

The  variability  in  the  estimates  of  probability  was  similar  across  the  different  materials, 
with  standard  deviations  ranging,  for  example,  from  17%  to  3 1%  for  estimates  of  P(A),  and 
comparably  for  P(B).  A  reviewer  suggested  that  some  participants  might  have  opted  for  an  easy 
solution  in  which  they  estimated  50%  for  all  three  probabilities,  but  in  fact  such  a  pattern  of 
responses  occurred  only  on  less  than  1%  of  trials.  In  general,  responses  of  50%  were  relatively 
rare  in  this  experiment  (and  the  subsequent  ones),  occurring  on  less  than  5%  of  estimates.  In 
order  to  assess  the  variability  in  the  participants’  responses,  we  computed  Shannon’s  measure  of 
infonnation  for  each  participant.  Its  overall  mean  was  0.90  bits,  and  it  varied  from  0.58  bits  in 
the  participant  with  the  least  variety  in  numerical  responses  to  1.10  bits  in  the  participant  the 
greater  variety  in  numerical  responses.  The  participants  differed  reliably  in  this  measure 
(Friedman  non-parametric  analysis  of  variance,  %“  =  60.33,  p  <  .001).  This  result  suggests  that 
participants  did  not  use  a  single  uniform  strategy  in  estimating  the  probabilities  of  conjunctions, 
and  they  did  not  make  trivial  errors,  such  as  estimates  of  50%  for  all  three  probabilities. 

Figure  2  presents  the  latencies  of  the  probability  judgments.  Not  surprisingly,  estimates 
of  the  conjunction  take  longer  than  estimates  of  its  conjuncts  (Wilcoxon  test,  z  =  5.07,  p  < 

.0001).  The  latencies  show  that  the  earlier  in  the  sequence  an  estimate  of  a  conjunction  is  made, 
the  longer  it  takes  (Page’s  L  =  562,  z  =  6.33,  p  <  .0001),  and  likewise  for  the  estimates  of  the 
conjuncts.  The  two  preceding  statistical  tests  depend  only  on  the  rank  order  of  latencies,  and  so  a 
transfonnation  of  the  raw  data,  say,  to  log  latencies,  would  have  no  effect  on  their  results.  They 
corroborate  the  model  theory’s  fourth  prediction.  When  the  conjunction  occurs  first  in  the  set  of 


estimates,  the  participants  think  about  both  A  and  B  in  an  intuitive  way  and  map  their  estimates 
onto  the  numerical  scale.  But,  when  the  judgments  of  P(A)  or  P(B)  precede  the  judgment  of  their 
conjunction,  its  estimate  can  take  into  account  the  numerical  estimates  for  the  conjuncts.  Given 
these  numerical  estimates,  individuals  are  also  more  likely  to  multiply  probabilities  and  to  avoid 
violations  of  the  JPD. 


Figure  2.  The  latencies  (s)  of  P(A),  P(B),  and  P(A&B)  in  Experiment  1  depending  on  the  order  in 
which  they  were  estimated.  Text  in  bold  shows  the  position  in  the  sequence  of  estimates  at  which 
the  relevant  estimate  was  made. 


P(A)  P(B) 


P(A&B) 


6.  Experiment  2:  Disjunctions  of  unique  events 

Intuitive  estimates  of  the  probabilities  of  inclusive  disjunctions  should  tend  to  be  based  on  the 
primitive  average  of  the  estimates  of  the  two  disjuncts,  and,  as  a  result,  to  violate  the  JPD.  In 
cases  in  which  participants  deliberate,  they  should  tend  to  add  the  estimates  of  the  two  disjuncts 


even  though  this  procedure  is  appropriate  only  for  mutually  exclusive  events.  Experiment  2 
tested  the  predictions  using  the  same  materials  as  those  in  the  previous  experiment,  but  the 
compound  was  now  a  disjunction: 

28.  What  is  the  chance  that  a  nuclear  weapon  will  be  used  in  a  terrorist  attack  in  the  next 

decade  or  there  will  be  a  substantial  decrease  in  terrorist  activity  in  the  next  10  years,  or 
both? 

6. 1  Method 

The  participants  each  had  to  estimate  sets  of  three  probabilities  of  unique  events  in  two 
different  orders:  P(A),  P(B),  P(AvB),  and  P(AvB),  P(A),  P(B).  They  carried  out  16  problems  in 
total.  The  order  of  the  problems  and  the  assignment  of  materials  were  randomized,  and 
participants  encountered  a  particular  material  only  once  in  the  experiment.  The  program  that 
administered  the  experiment  presented  each  question  separately,  and  it  recorded  the  participant’s 
numerical  estimate  and  its  latency.  The  4 1  participants  who  carried  out  the  experiment  were  a 
new  sample  from  the  same  population  as  before.  All  of  them  stated  that  they  were  native  English 
speakers.  They  participated  in  a  $10  lottery  as  in  Experiment  1. 

6.2  Results  and  discussion. 

The  participants  concurred  in  the  rank  order  of  their  estimates  over  the  different  materials 
for  P(A),  for  P(B),  and  for  P(AvB)  (Kendall’s  W’s  =  .29,  .17,  .18,  respectively,  p  <  .0001  in  all 
three  cases).  The  result  corroborates  the  model  theory’s  first  prediction  that  participants  rely  at 
least  in  part  on  knowledge  and  processes  in  common  to  make  estimates  of  unique  events.  Figure 
3  shows  the  percentages  of  trials  for  each  of  the  16  materials  in  which  participants  estimated  the 


probability  of  the  disjunctions  by  taking  p-averages  of  the  probabilities  of  their  disjuncts,  and  p- 
additions  of  the  probabilities  of  the  their  disjuncts.  An  additive  response  would  have  been 
nonnative  had  the  two  events  been  mutually  inconsistent,  but  they  were  not.  Goodness  of  fit 
metrics  showed  that  the  p-average  of  each  participants’  estimates  of  P(A)  and  P(B),  on  a  trial  by 
trial  basis,  yielded  a  better  fit  of  the  data  (Kohnogorov-Smirnov  test,  D  =  .29,  p  =  .06;  R  =  .60,  p 
<  .0001;  RMSE  =  .09)  than  the  p-addition  of  their  estimates  (Kohnogorov-Smirnov  test,  D  =  .71, 
p  <  .0001;  R2  =  .52,  p  <  .0001;  RMSE  =  .25).  The  best  fit  was  based  on  a  p-average  of  P(A)  and 
P(B)  for  violations  of  the  JPD  and  otherwise  their  p-addition  (Kohnogorov-Smirnov  test,  D  = 

.24,  n.s.;  R2  =  .77,  p  <  .0001;  RMSE  =  .11).  Overall,  32  of  the  41  participants  made  disjunctive 
estimates  that  fell  between  their  estimates  of  P(A)  and  P(B)  on  at  least  50%  of  trials  (Binomial 
test,  p  <  .0005).  They  also  bear  out  the  restricted  nature  of  system  1,  which  cannot  compute  the 
correct  value  of  P(A)  +  P(B)  -  P(A&B).  These  results  corroborate  the  model  theory’s  second 
prediction  that  individuals  should  make  systematic  violations  of  the  norms  of  the  probability 
calculus,  and  that  in  the  case  of  disjunctions  their  estimates  were  either  primitive  averages  or 
primitive  additions  of  the  two  probabilities. 

Figure  3:  The  percentages  of  trials  on  which  the  participants  in  Experiment  2  estimated  P(AvB) 
by  taking  a  p-average  or  a  p-addition  of  P(A)  and  P(B)  for  each  of  the  16  materials  in  the 
experiment  (see  Appendix  C).  The  balances  of  percentages  in  each  case  were  miscellaneous 


responses. 


Material  number 

Table  4  presents  the  percentages  of  trials  that  violated  the  JPD,  and  the  mean  percentages 
for  the  three  sorts  of  estimate.  Order  had  no  reliable  effect  on  the  frequency  of  JPD  violations 
(Wilcoxon  test,  z  =  1 .58,  p  =  .  12).  The  percentages  of  estimates  in  which  the  participants  added 
P(A)  and  P(B)  to  estimate  P(AvB)  were  as  follows: 

P(AvB)  P(A)  P(B)  3% 

P(A)  P(B)  P(AvB)  5% 

There  was  no  reliable  difference  between  the  two  orders  (Wilcoxon  test,  z  =  1 .29,  p  =  .20),  which 
is  contrary  to  the  third  prediction  of  the  model  theory.  But,  as  Figure  4  shows,  the  time  to  make 
an  estimate  speeded  up  over  the  triples  of  estimates  (Page’s  L  =  562,  z  =  6.33,  p  <  .0001).  The 
estimates  of  the  disjunction  took  longer  than  estimates  of  its  disjuncts  (Wilcoxon  test,  z  =  4.54,  p 
<  .0001),  and  longer  when  the  disjunction  occurred  first  than  when  it  occurred  last  (Wilcoxon 
test,  z  =  4.73,  p  <  .0001).  This  result  corroborates  the  model  theory’s  fourth  prediction. 


Table  4.  The  percentages  of  violations  of  the  JPD  in  Experiment  2,  and  the  mean  percentage 
judgments  for  three  sorts  of  events  depending  on  their  order.  Bolded  text  indicates  the  position  in 
the  sequence  of  estimates  at  which  the  relevant  estimate  was  made. 


The  order  of  the 


The  percentage  of  P(A)  as  a  P(B)  as  a  P(AvB)  as  a 


judgments 

violations  of  the 
JPD 

percentage 

percentage 

percentage 

1 .  P(AvB)  P(A)  P(B) 

62 

42 

37 

44 

2.  P(A)  P(B)  P(AvB) 

69 

41 

35 

42 

Overall  results 

65 

41 

36 

43 

As  in  the  previous  experiment,  we  computed  Shannon’s  measure  of  information  for  the 
estimates  of  each  participant.  Its  overall  mean  was  0.96  bits,  and  it  varied  from  0.76  bits  in  the 
participant  with  the  least  variety  in  numerical  responses  to  1 .07  bits  in  the  participant  the  greater 
variety  in  numerical  responses.  The  participants  differed  reliably  in  this  measure  (Friedman  non- 
parametric  analysis  of  variance,  yj  =  41.10,  p  =  .0002).  This  result  suggests  that  participants 
used  a  variety  of  strategies  in  estimating  disjunctive  probabilities.  In  line  with  this  result,  only  3 
out  of  the  41  participants  responded  50%  to  all  three  judgments  on  at  least  one  trial  in  the 
experiment. 

When  individuals  estimate  the  probability  of  the  disjunction  last,  they  have  access  to  their 
estimates  of  its  disjuncts.  However,  there  was  no  evidence  for  an  increase  in  additive  responses 
when  the  disjunction  occurred  last,  and  the  low  level  of  these  responses  is  surprising,  though 
probably  in  part  a  consequence  of  the  fact  that  on  nearly  a  third  of  trials,  a  sum  of  the 
probabilities  of  the  disjuncts  would  have  exceeded  a  probability  of  100%. 

Figure  4.  The  latencies  (s)  of  the  probability  estimates  in  Experiment  2  depending  on  their  order. 
Text  in  bold  shows  the  position  in  the  sequence  of  estimates  at  which  the  relevant  estimate  was 


made. 
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7.  Experiment  3:  Conditional  probability  estimates 

The  model  theory  postulates  that  individuals  estimate  the  probability  of  an  individual 
event,  P(B),  by  adducing  evidence.  It  treats  estimates  of  a  conditional  probability,  P(B\A),  in  an 
entirely  analogous  way  in  which  A  is  evidence  pertinent  to  P(B).  In  case  individuals  have  a  prior 
estimate  of  this  probability,  this  new  evidence  can  shift  this  estimate  one  way  or  another  (see 
sections  3.1  and  4.1).  The  process  can  lead  to  violations  of  the  JPD,  because  it  does  not  take  into 
account  either  P(A)  or  the  converse  conditional  probability.  Experiment  3  tested  the  predicted 
corollaries  of  this  account:  the  participants’  estimates  should  not  accord  with  Bayes’s  theorem, 
and  often  violate  the  JPD. 

7.1  Method 

The  participants  made  four  probability  estimates  concerning  two  unique  events  in  a  fixed 
order:  P(A),  P(B\A),  P(B),  and  P(A\B).  Examples  of  P(B\A)  and  P(A\B)  are  as  follows: 

29.  What  is  the  chance  that  there  will  be  a  substantial  decrease  in  terrorist  activity  in  the  next 
10  years  assuming  that  a  nuclear  weapon  will  be  used  in  a  terrorist  attack  in  the  next 


decade? 


30.  What  is  the  chance  that  a  nuclear  weapon  will  be  used  in  a  terrorist  attack  in  the  next 

decade  assuming  that  there  will  be  a  substantial  decrease  in  terrorist  activity  in  the  next 
10  years? 

The  sample  of  43  participants  was  from  the  same  population  as  before,  and  all  of  them  stated  that 
they  were  native  English  speakers.  They  participated  in  a  $10  lottery  as  in  Experiment  1.  They 
carried  out  12  problems  in  total:  6  had  materials  in  which  A  increased  the  value  of  P(B),  and  6 
had  materials  in  which  A  decreased  the  value  of  P(B)  according  to  the  previous  norming  study. 
The  experiment  accordingly  used  12  of  the  16  materials  in  Appendix  C  (see  Figure  5  for  a  list  of 
the  12  materials).  The  order  of  the  problems  was  randomized  for  each  participant.  The  program 
that  administered  the  experiment  presented  each  question  separately,  and  it  recorded  the 
participant’s  numerical  estimate  and  its  latency. 

7.2  Results  and  discussion 

The  participants  concurred  in  the  rank  order  of  the  probabilities  that  they  estimated  over 
the  different  contents  for  P(A),  for  P(B),  and  for  their  conditional  probabilities,  P(B\A)  and 
P(A\B)  (Kendall’s  Ws  =  .28,  .14,  .09,  .17,  respectively,  p  <  .0001  in  all  four  cases).  The  result 
corroborates  the  model  theory’s  first  prediction  that  participants  adduce  common  evidence  to 
make  estimates  of  unique  events,  and  it  suggests  that  participants  are  prepared  to  estimate 
conditional  probabilities  in  the  absence  of  base  rate  information,  because  they  estimated  P(B\A) 
before  they  estimated  P(A),  which  was  a  subsequent  judgment  on  each  trial. 

Goodness  of  fit  metrics  showed  that  the  model  theory  provided  a  better  fit  of  the  data 
(Kolmogorov-Smimov  test,  D  =  .26,  p  =  .  1 1 ;  R2  =  .55;  RMSE  =  .  10)  than  Bayes’s  theorem  or  its 


equivalent  ratio  P(A&B)/P(A)  (Kolmogorov-Smimov  test,  D  =  .28,  p  =  .07;  R2  =  .40;  RMSE  = 
.13).  But,  the  model  theory’s  fit  is  noticeably  poorer  than  in  the  case  of  conjunctions 
(Experiment  1)  and  disjunctions  (Experiment  2).  One  possibility  is  that  the  prior  nonning  study 
of  whether  A  increases  P(B)  or  decreases  P(B)  failed  to  do  justice  to  the  present  participants’ 
beliefs.  We  therefore  carried  out  an  analysis  in  which  we  examined  the  fit  taking  either  one-step 
up  or  one-step  down  from  each  participant’s  estimate  of  P(B)  depending  on  the  direction  of  the 
participant’s  estimate  of  P(B\A),  where  a  step  was  according  to  the  granularity  of  mReasoner  s 
icon,  i.e.,  a  step  of  approximately  12.5%  percent.  The  resulting  fit  with  the  theory  was 
comparable  to  the  results  of  the  previous  experiments  (Kolmogorov-Smimov  test,  D  =  .19,  p  = 
.45;  R2  =  .73,  p  <  .0001;  RMSE  =  .07).  Figure  5  shows  the  percentages  of  trials  for  each  of  the  16 
contents  in  which  participants  estimated  the  conditional  probability,  P(B\A),  by  adjusting  their 
estimate  of  P(B)  in  this  minimal  way  (categorized  as  such  if  a  participants’  response  fell  within 
12.5%  of  the  adjustment),  and  the  percentages  of  trials  on  which  their  judgments  were  in  accord 
with  Bayes’s  theorem.  The  resulting  fit  with  the  theory  was  comparable  to  the  results  of  the 
previous  experiments  (Kolmogorov-Smimov  test,  D  =  .19,  p  =  .45;  R“  =  .73,  p  <  .0001;  RMSE  = 
.07). 

Figure  5.  The  percentages  of  trials  on  which  the  participants  in  Experiment  3  estimated  P(B\A) 
by  a  minimal  adjustment  to  P(B)  and  the  percentages  in  accordance  with  Bayes’s  theorem  for 
each  of  the  12  contents  in  the  experiment  (see  Appendix  C).  The  balances  of  percentages  in  each 
case  were  miscellaneous  responses. 


Material  number 


Overall,  the  participants’  first  three  estimates  violated  the  JPD  on  24%  of  the  trials,  and 
42  out  of  the  43  participants  made  one  or  more  such  estimates  (Binomial  test,  p  <  .0001).  On 
each  trial,  the  participants’  first  three  estimates  -  of  P(A),  P(B\A),  P(B)  -  provide  the  input  for 
Bayes’s  theorem  to  predict  their  fourth  estimate  of  P(A\B),  and  so  we  examined  to  what  extent 
they  might  rely  on  the  theorem  to  make  this  fourth  estimate.  The  result  was  as  follows: 
(Kohnogorov-Smirnov  test,  D  =  .37,  p  <  .01;  R2  =  .02;  RMSE  =  1.25).  Hence,  the  correlation 
between  the  prediction  based  on  Bayes’s  theorem  and  the  participants’  estimates  was 
insignificant,  and  the  two  sets  of  data  were  reliably  different.  In  many  cases,  the  participants’ 
first  three  estimates  violated  the  JPD,  but  even  when  we  capped  the  estimates  to  prevent  such 
violations  the  correlation  with  Bayes’s  prediction  was  weak  (R“  =  .29).  These  results  corroborate 
the  model  theory’s  second  prediction  that  individuals  should  make  systematic  violations  of  the 
norms  of  the  probability  calculus. 

Table  5  presents  the  percentages  of  trials  that  violated  the  JPD,  and  the  mean  percentages 
of  the  four  sorts  of  estimate  depending  on  whether  A  increased  or  decreased  the  probability  of 


event  B.  This  difference,  estimated  in  the  prior  norming  study,  had  an  overall  reliable  effect  on 
estimates  of  P(B\A )  and  P(A\B)  (Wilcoxon  test,  z  =  2.15,  p  =  .03).  The  fixed  order  of  the  four 
estimates  -  P(A),  P(B\A),  P(B),  P(A\B)  did  not  allow  us  to  evaluate  the  model  theory’s  third 
prediction.  Informal  studies  have  sometimes  found  that  individuals  give  the  same  values  for  the 
two  conditional  probabilities  (Baron,  personal  communication).  Our  participants  did  so  on  15% 
of  trials. 

Table  6  presents  the  latencies  of  the  probability  judgments.  The  table  shows  that 
estimates  of  P(B\A)  take  longer  than  any  of  the  other  three  estimates  (Wilcoxon  tests,  zs  >  3.75, 
ps  <  .0005),  and  that  participants  are  faster  to  estimate  P(B)  than  P(A)  (Wilcoxon  test,  z  =  4.83,  p 
<  .0001).  The  data  are  consistent  with  the  model  theory’s  fourth  prediction  that  to  estimate 
P(B\A),  participants  tacitly  estimate  P(B).  The  result  is  to  speed  up  their  subsequent  estimate  of 
P(B). 


Table  5.  The  percentages  of  trials  violating  the  JPD  in  Experiment  3,  and  the  mean  percentage 
judgments  for  the  four  sorts  of  probability  depending  on  the  effect  of  A  on  P(B)  established  in  a 
prior  nonning  study. 


Effect  of  A  on 
P(B) 

The  percentage 
of  violations  of 
the  JPD 

P(A)  as  a 
percentage 

P(B  A)  as  a 
percentage 

P(B)  as  a 
percentage 

P(A  B)  as  a 
percentage 

A  increases  P(B) 

19 

42 

39 

43 

36 

A  decreases  P(B) 

28 

45 

30 

35 

42 

Overall  results 

24 

43 

35 

39 

39 

Table  6.  The  latencies  (s)  of  the  probability  estimates  in  Experiment  3  depending  on  the  effect  of 
A  on  P(B). 


Effect  of  A  on  P(B)  Estimates  of  Estimates  of  Estimates  of  Estimates  of 

_ P(A) _ P(B|A) _ P(B) _ P(A|B) 


A  increases  P(B) 

10.76 

11.54 

7.07 

10.11 

A  decreases  P(B) 

9.26 

12.07 

9.16 

9.87 

Overall  means 

10.01 

11.81 

8.12 

9.99 

In  general,  the  results  of  Experiment  3,  like  those  of  the  previous  experiments,  corroborated  the 
model  theory. 


8.  General  Discussion 

Why,  depending  on  the  odds,  do  people  accept  or  reject  bets  about  unique  events?  After 
all,  an  event  such  as  the  election  of  Hillary  Clinton  to  the  Presidency  will  either  occur  or  not  -  a 
fact  that  has  led  frequentists  to  declare  that  a  statement  of  its  probability  is  meaningless 
(Cosmides  &  Tooby,  1996,  p.  6)  or  that  the  probability  calculus  is  irrelevant  to  these  judgments 
(Gigerenzer,  1994).  A  Bayesian  riposte  is  that  the  probability  of  a  unique  event  reflects  the 
degree  to  which  an  individual  believes  in  its  occurrence,  i.e.,  the  event’s  subjective  probability. 
Hence,  granted  a  good  pay-off,  when  the  odds  of  a  bet  offered  by  a  bookmaker  translate  into  a 
probability  better  than  this  subjective  probability,  individuals  are  apt  to  accept  the  bet;  but  when 
the  odds  translate  into  a  probability  worse  than  this  subjective  probability,  they  are  apt  to  spurn  it 
(de  Finetti,  1936/1995;  Ramsey,  1926/1990).  The  riposte  reveals  three  mysteries  that  the  present 
paper  was  designed  to  solve:  the  source  of  the  numbers  in  subjective  probabilities,  the 
mechanism  that  determines  their  magnitudes,  and  the  process  yielding  the  probabilities  of 
compound  assertions,  such  as  disjunctions.  To  tackle  them,  our  research  has  introduced  several 
innovations.  Previous  studies  of  the  probabilities  of  compounds  reported  errors  in  their  estimates 
(e.g.,  Abelson,  Leddo,  &  Gross,  1987;  Zhao  et  al.,  2009),  but  did  not  provide  a  definitive 
explanation  of  them.  We  developed  a  dual-process  theory  postulating  intuitions  based  on 
uncertainty,  simulations  of  evidence  in  mental  models,  and  deliberations  that  translated  non- 


numerical  icons  into  numbers.  We  implemented  the  model  theory’s  account  of  system  1  and 
system  2  in  a  computer  program,  mReasoner,  which  integrates  deductive  and  probabilistic 
reasoning.  The  model  theory  applies  to  estimates  of  the  probabilities  of  unique  events,  and  to 
conjunctive,  disjunctive,  and  conditional  probabilities  of  them. 

The  model  theory  distinguishes  between  system  1  and  system  2  in  computational  power. 
System  1  has  a  severely  curtailed  power,  carrying  out  loops  of  operations  only  a  small  number  of 
times.  For  a  unique  event,  such  as: 

31.  What  is  the  probability  that  Hillary  Clinton  is  elected  US  President  in  2016? 
individuals  may  have  access  to  various  sorts  of  knowledge  pertinent  to  an  estimate  of  probability. 
They  may  adduce  knowledge  of  possibilities,  of  causal  relations,  and  of  other  sorts  of  factors, 
such  as: 

•  her  age. 

•  her  gender. 

•  her  political  career 

•  her  spouse 

•  possible  alternative  Democratic  candidates. 

•  possible  Republican  candidates. 

The  limitations  of  system  1  imply  that  individuals  tend  to  adduce  a  minimum  of  evidence,  but  it 
is  a  task  for  future  studies  to  investigate  the  number  and  the  factors  that  detennine  it.  The  theory 
implies  that  each  piece  of  evidence  is  considered  in  relation  to  general  knowledge.  Clinton,  for 
example,  will  be  older  than  that  of  most  US  presidents  when  they  were  elected  (a  median  age  of 
54  years),  and  so  this  renders  her  election  slightly  improbable.  The  model  theory  proposes  that 
the  evidence  is  used  to  set  an  initial  magnitude  of  belief,  and  to  modify  it  if  necessary  in  the  light 


of  subsequent  evidence.  In  mReasoner,  quantified  assertions  play  an  exemplary  role  in  yielding 
intuitive  probabilities.  For  example,  individuals  may  believe: 

32.  Few  candidates  over  the  age  of  60  years  are  elected  to  the  US  Presidency. 

But,  the  theory  does  not  imply  that  all  evidence  must  correspond  to  a  quantified  assertion.  Other 
evidence  can  be  conditional:  If  a  candidate’s  health  is  poor  then  he  or  she  may  not  be  elected.  A 
reviewer  suggested  that  evidence  itself  is  weighted  so  that  some  facts  play  a  bigger  role  than 
others  in  setting  a  probability.  Convincing  evidence  for  the  spontaneous  use  of  such  weightings 
would  be  contrary  to  the  model  theory,  because  a  weighting  mechanism  seems  beyond  the 
capability  of  system  1 . 

Given  the  preceding  evidence  (32),  the  program  simulates  it  in  a  mental  model,  such  as: 

33.  Over  60  in  age  elected  US  President 

Over  60  in  age 

Over  60  in  age 
Over  60  in  age 

The  proportion  of  cases  yields  an  analog  magnitude  in  an  icon  corresponding  to  a  degree  of 
belief  that  Clinton  is  elected: 

34.  |—  | 

The  deliberative  system  2  can  map  this  icon  into  a  numerical  magnitude,  such  as  a  probability  of 
30%. 

The  model  theory  postulates  that  intuitions  take  uncertainty  as  a  prime  sign  of 
improbability.  One  way  in  which  uncertainty  arises  is  when  a  conflict  occurs.  Suppose,  for 
example,  that  one  piece  of  evidence  suggests  that  the  event  in  (3 1)  is  unlikely,  whereas  another 
piece  of  evidence  suggests  that  it  is  likely.  System  1  resolves  the  conflict  with  a  compromise.  It 


takes  a  primitive  average,  and  infers  that  the  event  is  as  likely  as  not.  One  unforeseen  aspect  of 
the  implementation  in  mReasoner  is  the  simplicity  of  primitive  averaging  in  comparison  with 
other  primitive  operations  (see  Table  2). 

Conflicts  can  also  occur  in  compounds  of  events.  Given  that  no  logical  relations  exist 
between  two  unique  events,  A  and  B,  a  conflict  arises  in  the  probability  of  their  conjunction, 
when  system  1  yields  different  intuitive  probabilities  for  them.  System  1  again  resolves  the 
conflict  by  taking  their  primitive  average.  A  slightly  more  rational  response  in  the  light  of  the 
probability  calculus  is  to  take  their  primitive  product,  though  this  response  is  correct  only  if  one 
event  has  no  effect  on  the  probability  of  the  other.  Suppose,  now,  that  such  a  conflict  arises  in 
the  probability  of  an  inclusive  disjunction  of  two  unique  events,  A  or  B  or  both  -  again,  on  the 
assumption  that  there  are  no  logical  implications  between  them.  Disjunctions  yield  uncertainty 
(e.g.,  Bauer  &  Johnson-Laird,  1993;  Tversky  &  Shafir,  1992),  which  cues  improbability.  And  so, 
once  again,  system  1  uses  the  compromise  of  a  primitive  average.  A  slightly  more  rational 
response  in  this  case  is  to  sum  P(A)  and  P(B),  though  this  response  is  correct  only  if  A  and  B 
cannot  both  occur. 

Some  philosophers  and  psychologists  have  argued  that  individuals  estimate  a  conditional 
probability,  P(B\A),  using  a  modified  version  of  Ramsey’s  test  (e.g.,  Bennett,  2003;  Evans,  2007, 
p.  54).  You  add  A  to  your  beliefs.  If  you  know  that  A  is  false,  you  adjust  your  beliefs  where 
necessary  to  ensure  their  consistency,  and  then  you  estimate  the  probability  of  B.  The  problem  is 
that  individuals  have  too  many  beliefs,  and  so  a  test  of  their  consistency  is  intractable  (Johnson- 
Laird,  Girotto,  &  Legrenzi,  2004;  Oaksford  &  Chater,  2007,  p.  107).  Given  that  system  1  lacks 
computational  power,  intuitions  about  a  conditional  probability,  P(B\A),  are  likely  to  rely  on  a 
much  simpler  procedure.  System  1  treats  A  as  evidence  that  may  increase  P(B)  or  decrease  it 


(see  Section  3.1).  In  the  absence  of  a  logical  relation  between  them,  the  system  then  makes  a 
minimal  adjustment  -  in  terms  of  the  granularity  of  icons  -  in  order  to  revise  P(B).  If  A  has  no 
effect  on  the  probability  of  B ,  then  the  two  events  are  independent,  and  no  adjustment  to  P(B)  is 
necessary.  This  procedure  for  conditional  probabilities  has  a  family  resemblance  to  Ramsey’s 
test,  but,  as  its  implementation  in  mReasoner  shows  (see  Section  4. 1),  it  is  tractable  and  rapid 
enough  to  square  with  the  latencies  of  the  participants’  estimates  in  Experiment  3. 

The  use  of  intuitions,  as  the  model  theory  and  mReasoner  imply,  leads  to  numerical 
estimates  of  the  probabilities  of  compounds  that  should  tend  to  violate  the  probability  calculus. 

To  measure  the  degree  of  such  violations,  we  introduced  a  measure  based  on  negative 
probabilities  in  the  complete  joint  probability  distribution  (JPD),  where  the  JPD  for  two  events,  A 
and  B,  is  the  following  set  of  probabilities:  P(A&B),  P(A&^B),  P(^A  &B),  and  P(~'A&^B), 
where  denotes  negation  (see  the  Introduction).  The  predictions  with  one  exception  were 
corroborated  in  all  three  of  our  experiments: 

First,  the  participants  concurred,  weakly  but  reliably,  in  the  rank  order  of  their  estimates 
of  the  probabilities  of  unique  events,  and  their  compounds. 

Second,  they  tended  to  violate  the  JPD  in  the  ways  that  the  model  theory  predicts. 
According  to  the  probability  calculus,  the  probability  of  a  conjunction  of  events,  P(A&B),  is  the 
product  of  the  probability  of  one  event,  such  as  P(A),  and  the  conditional  probability  of  the  other 
event,  P(B\A).  This  computation  is  beyond  the  power  of  system  1,  and  so,  as  the  model  theory 
predicts,  individuals  tend  to  take  a  primitive  average  of  the  two  probabilities,  or  otherwise  to 
multiply  them  (Experiment  1).  The  average  yields  violations  of  the  JPD,  but  it  provided  a  better 
fit  to  the  data  than  a  simple  multiplicative  account.  The  best  fit  used  both  procedures.  The  use 
of  averaging  was  even  apparent  in  the  overall  means  in  four  of  the  problems.  For  instance,  the 


mean  estimate  of  the  conjunction  of  millions  of  people  living  past  100  and  an  end  to  the 
replacement  organ  shortage  was  22%,  a  value  falling  between  the  mean  estimates  of  the  two 
conjuncts  (19%  and  52%,  respectively),  and  these  means  violate  the  JPD.  The  probability  of  an 
inclusive  disjunction  of  events,  P(AvB),  equals  P(A)  +  P(B)  -  P(A&B).  This  computation  is  also 
beyond  the  power  of  system  1 ,  and  so  the  model  theory  predicts  that  the  uncertainty  evoked  by 
disjunctions  leads  individuals  to  average  the  probabilities  of  the  disjuncts.  The  primitive 
averaging  operation  yields  violations  of  the  JPD,  but  it  provided  a  better  fit  to  the  data  than  an 
additive  model  (Experiment  2).  The  best  fit  used  both  procedures,  though  the  addition  of 
probabilities  is  appropriate  only  if  the  disjunction  was  exclusive,  not  inclusive.  The  conditional 
probability,  P(B\A),  can  be  computed  according  to  Bayes’s  theorem  (3).  This  computation  is 
again  beyond  the  power  of  system  1 ,  and  so  too  is  the  equivalent  computation  of  the  ratio  of 
P(A&B)  to  P(A).  In  fact,  as  the  model  theory  predicts,  individuals  tend  to  adjust  P(B)  a  minimal 
amount  according  to  whether  ,4  increases  or  decreases  the  probability  of  B.  This  simple 
procedure  provided  a  better  fit  to  the  data  than  Bayes’s  theorem  (Experiment  3).  The  best  fit  of 
all  ignored  the  predicted  effect  of  A  on  P(B)  as  detennined  in  a  prior  norming  study,  and  simply 
adjusted  P(B)  up  or  down  by  a  minimal  amount.  In  other  words,  participants  seem  to  have 
differed  for  certain  contents  from  those  tested  in  the  nonning  study  about  the  effects  of  A  on 
P(B). 

Third,  estimates  of  the  probability  of  a  conjunction  were  more  likely  to  be  multiplicative 
after  the  participants  had  estimated  the  numerical  probabilities  of  their  conjuncts  (Experiment  1). 
The  one  exception  to  a  corroboration  of  the  model  theory  was  there  was  no  reliable  sign  of  a 
greater  tendency  for  additive  estimates  of  disjunctions  when  these  estimates  occurred  last  in  the 
series.  Very  few  participants  in  the  study  made  additive  estimates,  perhaps  because  many  such 


additions  would  have  exceeded  a  probability  of  100%. 

Fourth,  estimates  of  the  probabilities  of  conjunctions  and  disjunctions  were  faster  after 
estimates  of  their  constituents  (Experiments  1  and  2).  The  estimates  of  conditional  probabilities, 
P(B\A)  also  took  longer  than  the  estimates  of  P(A)  or  P(B),  and  the  estimate  of  the  conditional 
probability,  as  the  model  theory  predicts,  speeded  up  estimates  of  P(B)  (Experiment  3). 

The  role  of  deliberations  about  probabilities  (in  system  2)  is  curtailed  by  ignorance.  Most 
people  know  little  about  the  probability  calculus.  Readers  of  this  article  know  that  the 
probability  of  a  conjunction  cannot  be  larger  than  that  of  either  of  its  conjuncts,  and  that  the 
probability  of  an  inclusive  disjunction  cannot  be  smaller  than  that  of  either  of  its  disjuncts.  They 
are  less  aware,  we  suspect,  of  the  third  constraint  on  each  of  these  estimates  (see  Appendix  B): 
given  P(A)  and  P(B)  an  estimate  of  P(A&B)  can  be  too  low,  and  an  estimate  of  P(AvB)  can  be  too 
high.  The  constraints  on  conditional  probabilities  are  still  more  recondite.  Before  we  started  this 
research,  our  guess  would  have  been  that  P(A)  and  P(B)  put  no  constraints  on  P(A  \B)  in  order  to 
yield  a  consistent  JPD.  Even  experts  do  not  know  at  once  what  the  constraints  are,  but  have  to 
calculate  them  (see  Appendix  A).  We  can  illustrate  the  impotence  of  system  2  with  two 
problems.  The  first  problem  is: 

34.  If  P(A)  =  .7  and  P(B)  =  .3,  what  is  the  range  of  values  for  the  conditional  probability 
P(A  | B)  that  yield  a  consistent  JPD? 

The  second  problem  has  the  opposite  values: 

35.  If  P(A)  =  .3  and  P(B)  =  .7,  what  is  the  range  of  values  for  P(A \B)  that  yield  a  consistent 
JPD? 

The  exact  numerical  values  call  for  calculation,  but  which  of  the  two  problems  allows  a  greater 
range  of  values  for  P(A \B)1  The  answer  is  the  first  problem,  which  for  a  consistent  JPD  allows 


that  the  conditional  probability  can  be  anywhere  from  0  to  1,  whereas  for  the  second  problem  it 
must  be  in  the  narrower  range  from  0  to  .42  to  two  decimal  places.  Even  the  use  of  the  Ramsey 
test  to  estimate  conditional  probabilities  overlooks  the  constraint  that  P(A)  can  put  on  P(B\A). 

Skeptics  might  argue  that  our  participants  are  not  making  real  estimates  of  probabilities. 
They  are  instead  indulging  in  unprincipled  guesses.  Evidence  counts  against  this  conjecture.  The 
participants  perfonned  better  than  chance  in  all  three  experiments  (see  Appendix  B  for  the 
chance  values).  They  concurred  about  the  relative  probabilities  of  the  different  contents,  and 
these  reliable  concordances  in  all  three  experiments  imply  that  they  relied  to  some  degree  on 
common  knowledge.  The  latencies  of  responses  showed  systematic  patterns,  reflecting  greater 
thought  about  compounds  than  about  their  constituent  events,  and  faster  responses  to  compounds 
when  the  participants  had  already  assessed  the  numerical  probabilities  of  their  constituents. 
Indeed,  the  robust  fit  of  the  processes  implemented  in  mReasoner  to  the  patterns  of  data  could 
not  have  occurred  without  the  participants’  reliance  on  systematic  mental  processes. 

Other  skeptics  might  argue  that  alternative  theories  could  account  for  our  results,  and  that 
no  need  exists  to  postulate  a  dual-process  theory  or  processes  based  on  mental  models.  For 
example,  an  alternative  theory  might  posit  that  degrees  of  belief  depend  on  activations  of 
particular  chunks  in  memory  (Anderson,  1990),  or  else  reflect  the  causal  power  of  instances  of 
one  concept  on  those  of  another  (e.g.,  Cheng,  1997).  Of  course,  no  results  can  ever  show  that  a 
theory  is  the  only  possible  one,  and  so  clearly  other  theories  might  be  framed  to  explain  the 
phenomena.  It  is  critical,  however,  that  any  alternative  theory  accounts  for  the  origins  of 
numerical  estimates  from  non-numerical  premises,  for  how  this  information  is  mentally 
represented,  and  for  why  estimates  violate  the  JPD.  Accounts  of  reasoning  based  on  fonnal  rules 
of  inference  (e.g.,  Rips,  1994)  offer  no  account  of  quantifiers,  such  as  “most”  and  “few”,  and  so 


they  cannot  explain  their  role  in  probabilistic  thinking.  Perhaps  surprisingly,  theories  of 
reasoning  based  on  probability  logic  (e.g.,  Oaksford  and  Chater,  2007)  offer  no  account  of  how 
numerical  probabilities  derive  from  non-numerical  considerations.  Hence,  at  present,  the  model 
theory  offers  the  only  explanation  of  the  phenomena,  it  is  implemented  in  a  computer  program, 
and  its  predictions  emerge  from  its  unification  of  deductive  and  probabilistic  reasoning.  We  have 
previously  compared  the  model  theory’s  predictions  for  conjunctions  with  other  extant  theories 
of  probabilistic  estimates  (Khemlani  et  al.,  2012),  though  these  alternative  theories  were  not 
framed  for  the  probabilities  of  unique  possibilities  (Fantino  et  al.,  1997;  Wyer,  1976).  The 
comparison  showed  that  the  computational  implementation  of  mReasoner  yields  a  better  fit  to 
data  about  the  probabilities  of  conjunctions  than  these  alternatives. 

The  present  results  have  implications  for  the  role  that  probabilities  play  in  higher 
cognition.  Marcus  and  Davis  (2013)  have  shown  how  the  apparent  successes  of  probabilistic 
approaches  to  a  variety  of  tasks  -  from  categorization  to  inferences  in  naive  physics  -  may 
depend  on  investigators’  bias  selecting  certain  tasks  and  neglecting  others,  and  also  on  the  choice 
of  different  statistical  models  from  one  domain  to  another.  In  the  case  of  reasoning,  recent 
developments  -  often  motivated  by  the  goal  of  sustaining  the  rationality  of  naive  individuals  - 
have  led  to  a  cumulative  introduction  of  probabilities  into  accounts  of  simple  classical 
deductions.  A  modest  step  is  to  argue  that  the  meaning  of  a  conditional  assertion,  such  as: 

36.  If  Clinton  is  elected  US  President  in  2016  then  all  wars  in  the  world  will  cease  thereafter, 
is  a  conditional  probability  (e.g.,  Adams,  1998;  Douven  &  Verbrugge,  2013;  Pfeifer,  2013; 

Pfeifer  &  Kleiter,  2005).  The  next  step  replaces  deductive  validity  with  probabilistic  validity, 
where  an  inference  is  probabilistically  valid  provided  that  the  infonnativeness  of  its  conclusion 
(1  minus  its  probability)  is  not  greater  than  sum  of  the  premises’  infonnativeness  (Adams,  1998). 


One  proponent  of  a  probabilistic  approach  to  deduction,  however,  proposes  to  abandon  its 
nonnative  evaluation  in  favor  of  description  alone  (Evans,  2012).  Others  argue:  “people  are  far 
better  probabilistic  reasoners  than  Tversky  and  Kahneman  supposed”  (Oaksford  &  Chater,  1998, 
p.  262).  The  burden  of  the  present  results  on  violations  of  the  JPD  is  that  not  only  were  Tversky 
and  Kahneman  correct,  but  that  human  competence  is  still  worse  than  one  might  have  supposed: 
errors  occur  even  when  inferences  about  probabilities  are  not  elicited  in  scenarios  likely  to 
trigger  heuristics  such  as  representativeness. 

The  most  radical  probabilistic  proposal  replaces  the  classical  probability  calculus  with 
quantum  probabilities,  which  treat  JPDs  with  negative  values  as  acceptable,  and  so  every 
response  in  our  experiments  is  rational.  But,  if  psychologists  abandon  all  nonns  of  rationality,  or 
equivalently  allow  that  any  set  of  estimates  of  probabilities  is  rational,  no  way  exists  to  assess 
competence.  Yet,  people  differ  in  the  ability  to  reason.  The  Wright  brothers  were  better  thinkers 
about  aeronautical  matters  than  their  rivals,  and  their  ability  to  think  of  counterexamples  to  their 
own  conjectures  helped  them  to  develop  the  first  controllable  heaver-than-air  craft  (Johnson- 
Laird,  2006,  Ch.  25).  Likewise,  John  Snow  was  a  better  thinker  about  the  mode  of 
communication  of  cholera  than  the  proponents  of  the  rival  “miasma”  theory.  He  understood  that 
the  gas  laws  refuted  communication  by  way  of  a  miasma  (Ibid.,  Ch.  27).  Yet,  a  single 
counterexample  as  a  decisive  refutation  plays  no  role  in  the  approaches  to  reasoning  summarized 
above,  and  so  the  yardstick  distinguishing  between  good  and  bad  thinking  is  in  danger  of 
disappearing  too. 

Let  us  put  aside  issues  of  rationality,  and  ask  what  is  certain  in  the  light  of  our  research. 

It  establishes  five  points  about  naive  probabilities: 

1 .  Individuals  readily  make  numerical  estimates  of  the  probabilities  of  unique  events. 


2.  Their  estimates  yield  systematic  violations  of  the  probability  calculus. 

3.  These  violations  are  predictable  in  tenns  of  the  intuition  that  uncertainty  cues 
improbability. 

4.  The  implementation  of  this  principle  in  a  particular  dual-process  algorithm  predicts  the 
violations.  Of  course,  there  are  infinitely  many  other  possible  algorithms  that  can  also  do 
so  in  principle  -  though  as  far  as  we  know  no  alternatives  yet  exist. 

5.  If  intuitions  about  probabilities  underlie  deductive  reasoning  then  people  will  err  in 
making  deductions  too,  even  granted  the  yardstick  of  probabilistic  validity. 

How  might  individuals  avoid  violations  of  the  probability  calculus  in  estimating  the  probabilities 
of  unique  possibilities?  Conjunctions  seem  transparent,  and  perhaps  the  best  way  to  reveal  their 
true  nature  is  to  consider  the  conjunction  of  very  rare  events,  such  as  the  probability  of  winning 
the  New  York  lottery  and  being  struck  by  lightning  on  the  same  day.  Individuals  may  realize  that 
such  a  conjunction  is  less  probable  than  either  of  its  conjuncts.  Disjunction  creates  a  veil  of 
uncertainty,  and  one  way  to  lift  the  veil  is  to  use  disjunctions  of  mutually  exclusive  events,  which 
reduce  the  number  of  mental  models.  Deductions  from  exclusive  disjunctions  are  easier  than 
those  from  inclusive  disjunctions  (e.g.,  Bauer  &  Johnson-Laird,  1993;  Johnson-Laird  et  ah, 

1992),  and  so  too  should  be  inferences  of  their  probabilities.  Individuals  should  be  more  likely 
to  make  a  primitive  addition  of  probabilities.  Still  easier  should  be  exclusive  disjunctions  that 
are  clearly  exhaustive,  such  as  the  probability  that  a  card  is  either  an  ace  or  not. 

At  the  end  of  Section  4.3,  we  raised  the  problem  of  a  set  of  three  probabilities  that  fix  the 
JPD  for  two  binary  variables  A  and  B,  but  whose  values  can  never  violate  it.  A  simple  case  is  an 
instance  of  the  law  of  total  probability  (e.g.,  Gnedenko  &  Khinchin,  1961,  p.  31): 

P(A)  =  P(A|B).P(B)  +  P(A|-B).  P(-  B) 


If  we  substitute  b  for  P(B),  x  for  P(A  |  B),  and  y  for  P(A  |  B)  then: 

P(A)  =  xb  +  y(l  -  b) 

The  values  of  b,  x,  and  y,  are  free  to  vary  in  the  unit  interval  [0,  1],  and  they  fix  the  value  of 
P(A).  Hence,  no  matter  what  their  values  are  the  following  three  probabilities  fix  the  JPD  and 
cannot  violate  it: 

P(B),  P(A|B),  and  P(A|-  B). 

We  owe  this  argument  to  an  anonymous  reviewer.  There  are  other  answers  too,  e.g.,  the  three 
conditional  probabilities  also  fix  the  JPD  and  cannot  violate  it: 

P(B|A),  P(A|B),  and  P(A|~B). 

In  conclusion,  the  intuitions  of  naive  individuals  accord  with  the  Bayesian  interpretation 
of  probabilities  as  degrees  of  belief.  This  interpretation  extends  to  unique  events,  which  are 
important  in  daily  life  and  in  science.  The  relative  rarity  of  compound  bets  in  prediction  markets 
may  reflect  folk  wisdom  about  the  difficulty  of  confonning  to  the  probability  calculus,  and  hence 
a  potential  vulnerability  to  “Dutch  books”.  The  model  theory,  which  bases  rationality  on 
computations  that  simulate  the  world  (Craik,  1943),  explains  this  difficulty  in  terms  of  the 
restricted  operations  of  the  intuitive  system  of  reasoning.  They  curtail  the  ways  in  which  the 
system  can  combine  evidence  and  estimate  conjunctive,  disjunctive,  and  conditional 
probabilities. 
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Appendix  A:  The  constraints  on  conditional  probabilities 

Depending  on  the  values  of  P(A)  and  P(B),  sometimes  there  are  no  constraints  on  P(A  \B)  other 
than  it  is  in  the  unit  interval  [0,  1]  ,  sometimes  P(A\B)  must  be  larger  than  a  certain  value  greater 
than  0,  sometimes  it  must  be  smaller  than  a  certain  value  less  than  1,  and  sometimes  it  must  lie 
between  two  such  values,  in  order  to  yield  a  coherent  JPD.  With  higher  values  of  P(B),  the 
narrower  the  constraints.  One  way  to  derive  the  bounds  is  from  the  formulas  for  each 
component  of  the  JPD  given  P(A),  P(B '),  and  P(A\B): 

P(A&B)  =  P(B)  .  P(A|B) 

P(A&~B)  =  P(A)  -  (P(B)  .  P(A|B)) 

P(~A&B)  =  P(B)  -  (P(B)  .  P(A|B)) 

P(-A&-B)  =  (1  +  (P(B)  .  (P(A|B)  -  1)))  -  P(A) 

Each  of  these  values  must  be  in  the  unit  interval  [0,  1]  and  the  resulting  inequalities  from  these 
four  formulas  simplify  to  two: 

P(A)  >  (P(B)  .  P(A|B))  >  (P(A)  +  P(B)  -  1) 

A  more  general  and  rigorous  derivation,  which  we  owe  to  personal  communications  from 
Osherson,  Kleiter,  and  Gilio,  5/26/13  (see  also  Wallmann  &  Kleiter,  2013),  is: 

If  P(A)  =  x,  P(B)  =  y,  then  max{0,  x  +  y  -  1/x}  <  P(B|A)  <  min{  1,  y/x}. 

In  standard  treatments  of  conditional  probabilities,  P(B\A)  is  undefined  in  case  P(A)  =  0,  but  in 
treatments  in  which  conditional  probabilities  are  treated  as  undefined  primitives,  as  in  the 
analysis  due  to  de  Finetti  (1970),  the  value  of  P(B\A)  can  be  anywhere  in  the  unit  interval  in  this 


case. 


Appendix  B:  Formulas  and  constraints  of  the  complete  joint  probability  distributions 


Table  Bl:  The  formulas  for  the  complete  joint  probability  distributions  (JPD),  the  constraints  that 
they  yield  on  the  values  of  triples  for  conjunctive,  disjunctive,  and  conditional  probabilities  for 
consistent  JPDs,  and  the  percentages  of  0,  1,  and  2  violations  of  the  JPDs  from  exhaustive 
searches  of  possible  values  for  the  different  triples  of  probability  judgments.  Note:  denotes 

negation,  “&”  denotes  conjunction,  “v”  denotes  inclusive  disjunction,  and  “|”  denotes  a 
conditional  probability.  1%  of  instances  of  cases  of  conditional  probabilities  are  undefined 
because  of  potential  divisions  by  0,  and  those  instances  were  dropped  from  the  table. 


Chance 

The  formulas  for  the  JPD 

The  constraints  to  be  satisfied 
for  a  consistent  JPD 

percentages  of  n 
violations  of  the 

JPD 

0 

1 

2 

Conjunctions:  P(A),  P(B),  P(A&B) 

P(A&B)  =  given 

P(A&-B)  =  P(A)  -  P(A&B) 

P(-A&B)  =  P(B)  -  P(A&B) 

P(-A&-B)  =  1  -  (P(A)+P(B)  -  P(A&B)) 

P(A)  >  P(A&B) 

P(B)  >  P(A&B) 

1.0  >  P(A)  +  P(B)  -  P(A&B) 

17 

50 

33 

Disjunctions :  P(A),  P(B),  P(AvB) 

P(-A&-B)  =  1  -  P(AvB) 

P(-A&B)  =  (1  -  P( A))  -  P(-A&-B) 

P(A&B)  =  P(B)  -  P(-A&B) 

P(A&-B)  =  P(A)  -  P(A&B) 

P(AvB)  >  P(A) 

P(AvB)  >  P(B) 

P(B)  >  P(A)-P(AvB) 

17 

50 

33 

Conditional  probabilities:  P(A),  P(B),  P(B\A) 

P(A&B)  =  P(A)  *  P(B|A) 

P(A&-B)  =  P(A)  -  P(A)  *  P(B|A) 

P(-A&B)  =  P(B)  -  P(A)  *  P(B|A) 

P(-A&-B)  =  1  -  (P(A&B)  +  P(A&-B) 

+  P(-A&B)) 

P(B)  >  P(A)  *  P(B|A) 

P(A)  *  P(B|A)  >  P(A)  +  P(B)  -  1 

50 

50 

0 

Appendix  C:  The  16  sets  of  materials  used  in  the  three  experiments  were  based  on  the  following  conjunctions,  where  the  first 


conjunct  is  A  and  the  second  conjunct  is  B  in  the  estimates  of  probabilities. 


Material 

number 

1 

2 

3 

4 

5 

6 


Assertion  1 

What  is  the  probability  that... 

space  tourism  will  achieve  widespread  popularity  in 
the  next  50  years? 

in  less  than  1 5  years,  millions  of  people  will  live  past 
100? 

the  Supreme  Court  rules  on  the  constitutionality  of 
gay  marriage  in  the  next  5  years? 

Greece  will  make  a  full  economic  recovery  in  the  next 
10  years? 

a  nuclear  weapon  will  be  used  in  a  terrorist  attack  in 
the  next  decade? 

the  United  States  will  sign  the  Kyoto  Protocol  and 
commit  to  reducing  C02  emissions  in  next  1 5  years? 


Assertion  2 

What  is  the  probability  that... 

advances  in  material  science  will  lead  to  the 
development  of  anti-gravity  materials  in  the  next  50 
years? 

advances  in  genetics  will  end  the  shortage  of 
replacement  organs  in  the  next  15  years? 

that  a  gay  person  will  be  elected  as  president  in  the 
next  50  years? 

Greece  will  be  forced  to  leave  the  EU? 

there  will  be  a  substantial  decrease  in  terrorist  activity 
in  the  next  10  years? 

global  temperatures  reach  a  theoretical  point  of  no 
return  in  the  next  100  years? 


7 

8 

9 


intelligent  alien  life  is  found  outside  the  solar  system 
in  the  next  10  years? 

the  United  States  adopts  an  open  border  policy  of 
universal  acceptance? 

Honda  will  go  bankrupt  in  2012? 


world  governments  dedicate  more  resources  to 
contacting  extra-terrestrials? 

English  is  legally  declared  the  official  language  of  the 
United  States? 

Ford  will  go  bankrupt  before  the  end  of  2013? 


10 


a  significant  upturn  in  the  economy  occurs  next  year?  Obama  will  be  reelected  President  in  2012? 


11 

12 

13 


14 

15 


a  new  illegal  but  synthetic  drug  becomes  popular  in 
the  USA  over  the  next  two  years? 

US  companies  focus  their  advertising  on  the  Web  next 
year? 

3 -dimensional  graphics  will  be  required  to  contain 
explicit  markers  to  indicate  their  unreal  nature  by 
2020 

intellectual  property  law  in  the  US  will  be  updated  to 
a  reflect  advances  in  technology  by  the  year  2040? 

at  least  one  head  of  state  will  be  assassinated  by 
2012? 


the  movement  to  decriminalize  drugs  doubles  its 
numbers  by  2015? 

the  New  York  Times  becomes  more  profitable? 

competitive  video  game  playing  will  achieve 
mainstream  acceptance  by  2020? 

Russia  will  become  the  world  center  for  software 
development  by  2040? 

NATO  will  grant  military  support  to  Arab  Spring 
movements  in  several  countries? 


16 


scientists  will  discover  a  cure  for  Parkinson’s  disease 
in  10  years? 


the  number  of  patients  who  suffer  from  Parkinson’s 
disease  will  triple  by  2050? _ 


Note:  Data  from  Experiments  1,  2,  and  3  were  collected  in  2010  and  2011.  Several  of  the  materials  reference  years  that  have  since 
passed  and  therefore  have  definitive  answers,  but  they  were  controversial  for  the  participants  in  our  experiments. 


